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1.  INTRODUCTION 


1.1  STATEMENT  OF  THE  PROBLEM 

Ninety  percent  of  all  deaths  in  wartime  occur  on  the  battlefield,  before  a  casualty  has 
reached  a  surgeon  (killed  in  action,  or  KIA).  Rapid  evacuation  has  been  beneficial  in  decreasing 
the  number  of  casualties  dying  of  wounds  (DOW)  after  reaching  a  field  hospital,  but  has  not 
significantly  reduced  the  KIA  rate.  The  most  likely  reason  that  battlefield  mortality  remains  high 
is  that  definitive  surgical  care  is  not  available  during  the  critical  first  hour  after  wounding. 

The  effectiveness  of  trauma  care  for  woimded  combatants  is  also  limited  by  the  relatively 
small  number  of  surgeons  with  adequate  experience  to  treat  their  traumatic  injiuies.  Training  for 
military  combat  surgeons  is  currently  based  primarily  on  civilian  trauma  surgery  experience, 
supplemented  by  special  post-resident  military  training  [Bowen  and  Bellamy,  1988].  Because  of 
prevailing  peacetime  conditions,  fewer  and  fewer  military  medical  corpsmen  and  surgeons  have 
adequate  experience  with  the  types  of  battlefield  wovmds  they  will  encounter  in  the  line  of  duty. 

SRI  has  developed  a  revolutionary  new  technology — ^telepresence  surgery  — ^that  can 
address  both  of  these  problems.  To  address  the  need  for  more  rapid  intervention,  telepresence 
can  enable  surgeons  located  at  field  hospitals  to  directly  and  immediately  treat  wounded 
combatants  at  the  most  forward  echelons  of  care,  thus  gaining  precious  minutes  otherwise  lost 
during  evacuation.  To  improve  surgeon  training,  SRI’s  telepresence  technology  can  also  be  used 
as  an  advanced  surgical  simulation  and  training  tool  that  provides  a  realistic  haptic  and  visual 
training  medium  for  practicing  combat  surgery  in  peacetime  conditions.  Through  these  two 
avenues,  telepresence  has  the  potential  to  significantly  reduce  the  number  of  U.S.  service 
members  that  are  KIA. 

The  strength  of  SRI’s  telepresence  technology  derives  from  its  unique  integration  of 
high-fidelity  video,  audio,  and  haptic  (tactile)  sensory  inputs  to  the  surgeon  located  at  a  hospital 
remote  from  the  casualty,  and  the  real-time,  high-fidelity  output  to  the  remote  surgical  unit  at  the 
casualty’s  location,  enabling  precise  and  accurate  surgical  procedures  to  be  accomplished  by  the 
telepresence  surgeon.  The  same  high-fidelity  sensory  feedback  allows  a  telepresence  operator’s 
console  to  serve  as  a  highly  effective  user  interface  for  interacting  with  computer-based 
simulation  and  training  models. 

1 .2  PROJECT  GOALS  AND  METHOD  OF  APPROACH 

SRI  has  completed  three  closely  related  projects  for  DARPA  imder  the  referenced  grant, 
all  of  which  are  covered  by  this  final  report.  The  projects  are  (1)  Development  of  an  Advanced 
Telepresence  Surgery  System,  (2)  Telepresence  Surgery  Simulation,  and  (3)  Development  of  a 
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Testbed  for  Telepresence  Surgery  at  the  Uniformed  Services  University  of  the  Health  Sciences 
(USUHS). 

The  scope  of  the  first  project,  Development  of  an  Advanced  Telepresence  Surgery 
System,  was  to  develop,  fabricate,  deliver,  and  demonstrate  a  prototype  advanced  telepresence 
surgery  system  that  could  allow  a  surgeon,  located  in  a  mobile  army  surgical  hospital  (MASH) 
unit  or  base  hospital,  to  carry  out  emergency  surgical  procedures  on  soldiers  in  an  armored 
mobile  surgical  vehicle  in  the  combat  zone.  The  completed  system  consists  of  three  major 
components: 

•  A  mobile  remote  surgery  unit  (RSU)  installed  in  an  armored  vehicle.  Dubbed  the 
Medical  Emergency  Forward  Area  Surgical  Telepresence  (MEDFAST)  system,  it 
comprises  a  medic-positionable  pod  above  the  surgical  table,  on  which  are  mounted 
two  6-degree-of-fi:eedom  (DOF)  slave  manipulators  with  interchangeable  surgical 
instruments,  a  pair  of  high-resolution  video  cameras  (to  provide  a  stereoscopic  image 
pair),  and  additional  (low-bandwidth)  video  cameras  to  provide  a  panoramic  view  of 
the  MEDFAST  vehicle  interior. 

•  A  telepresence  surgeon’s  workstation  (TSW).  The  TSW  includes  a  pair  of  6-DOF 
force-feedback  master  manipulators,  a  high-resolution  stereographic  display  for 
viewing  the  surgical  field,  and  auxiliary  monitors  for  viewing  the  assisting  medic  and 
the  inside  of  the  MEDFAST  vehicle. 

•  A  high-data-rate,  low-latency,  two-way,  digital,  microwave  commimication  link 
between  the  MEDFAST  and  the  TSW.  The  communication  link  operates  over  a 
distance  of  up  to  5  km  and  includes  subsystems  for  video  digitization,  compression, 
and  encoding  of  all  signals  (video,  servo  commands,  audio,  patient  monitoring 
information),  as  well  as  modulation,  demodulation,  and  microwave  transmission. 

The  technology  and  systems  developed  as  part  of  this  MEDFAST  development  effort 
also  played  key  roles  in  the  Simulation  and  Testbed  projects:  An  identical  TSW  (with  master 
manipulators  and  stereo  display)  is  used  as  the  surgeon’s  interface  in  the  Surgical  Simulation 
System,  where  it  provides  a  highly  realistic  and  effective  haptic  and  visual  training  interface.  In 
the  Telepresence  Testbed  Project,  the  new  MEDFAST  technology  forms  the  basis  for  the 
Telepresence  Surgery  System  to  be  installed  at  USUHS. 

The  scope  of  the  second  project.  Telepresence  Surgery  Simulation,  was  to  adapt  and 
expand  the  Telepresence  Surgery  System  so  that  it  could  serve  as  a  surgeon’s  interface  to  virtual 
reality  (VR)  based  surgical  simulation  and  training  models,  and  to  provide  a  software  and 
hardware  platform  that  would  allow  surgeons  to  test,  evaluate,  and  guide  development  of 
simulation  models.  Specifically,  SRI  was  to  develop  a  surgical  simulation  and  training  system 
consisting  of 

•  A  complete  TSW,  including  master  manipulators,  stereo  display,  and  audio  feedback 

•  Servo  control  electronics,  adapted  for  simulation 
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•  Interface  software  coupling  master  manipulators  to  surgical  simulation  software 

•  Simulation  engine  for  generating  stereo  display  and  physical  modeling 

•  Simulation  and  training  models  imported  from  MusculoGraphics,  Inc.  (a  cooperating 
contractor  under  the  DARPA  program) 

The  scope  of  the  third  project,  Development  of  a  Testbed  for  Telepresence  Surgery  at 
USUHS,  was  to  provide  USUHS  with  a  complete  6-DOF  MEDFAST  system  comprising  a  TSW, 
RSU,  and  system  control  electronics.  SRI  was  to  fabricate  and  assemble  the  system,  install  it  at 
USUHS,  provide  technical  support  for  system  operation  and  maintenance,  and  install  system 
upgrades  that  were  subsequently  developed  under  the  grant. 

1.3  BACKGROUND 

The  application  of  robotic  technologies  in  surgery  has  grown  rapidly  in  the  last  few  years. 
Several  groups  have  developed  “image-guided”  surgery  systems  that  employ  highly  accurate 
robotic  arms  in  combination  with  high-resolution  image  data  (computed  tomography  [CT]  and 
magnetic  resonance  [MR])  to  perform  precise  svugical  positioning  and  cutting  under  computer 
control  [Dario  et  al.,  1996].  For  example,  a  specialized  robotic  system  (MINERVA)  was  recently 
developed  for  stereotactic  brain  surgery,  and  conventional  industrial  robots  have  been  used  for 
placement  of  probes  in  the  brain,  eye,  and  spine  [Burckhardt  et  al.,  1995].  Typically,  these 
computer-controlled  robots  insert  a  needle  or  other  probe  along  its  axis  deep  into  a  tumor,  guided 
by  skeletal  structures  and  other  positioning  landmarks  obtained  from  the  CT  and  MR  images. 

The  “Robodoc”  orthopaedic  surgery  system  [Taylor  et  al.,  1994],  developed  at  IBM  and 
Integrated  Surgical  Systems,  is  an  image-guided  system  that  functions  like  a  CNC  milling 
machine,  producing  a  precise  cavity  in  the  femur  for  hip  replacement.  Also  in  the  orthopaedic 
area,  Kienzle  et  al.  [1995]  have  developed  a  prototype  computer-controlled  system  for  use  in 
total  knee  joint  replacement.  Transurethral  resection  of  the  prostate  (TURP)  has  also  been 
performed  robotically  [Ng  et  al.,  1993]. 

In  contrast  to  the  robotic  automation  employed  in  image  guided  surgery,  others  have 
envisioned  the  potential  advantages  of  telemanipulation  in  surgery.  Alexander  [1978]  shows  a 
concept  for  a  teleoperator  surgery  system  in  which  the  operator  could  see  and  feel  what  he  or  she 
is  operating  on.  Thring  [1983]  describes  a  bilateral  surgical  telerobot  concept  in  which 
exoskeletal  masters  would  be  used  to  operate  a  pair  of  xyz  Cartesian  manipulators. 

Marsuhima  and  Koyanagi  [1981]  describe  a  handheld  master  with  a  tool-like  portion,  and 
Sabatini  et  al.  [1989]  describe  cutting  with  a  finger-sized  manipulator  (four  links)  proposed  for 
comeal  transplant  surgery.  Gayed  et  al.  [1987]  and  Guerrouad  and  Vidal  [1989]  have  described 
the  use  of  a  stereotaxic  manipulator  with  a  remote  center  of  rotation,  for  placing  a  probe  into  the 
eye.  They  used  a  6-axis  manipulator  with  joystick  controls  that  lacked  force  feedback.  Charles  et 
al.  [1989]  describe  the  human-interface  requirements  for  microsurgical  telerobot  surgery, 
Schenker  et  al.  [1994]  describe  a  robotically  assisted  microsurgery  system  (RAMS)  aimed  at 
enhancing  surgeon  dexterity  in  microsurgical  procedures,  and  Hill  tmd  Jensen  [1998]  describe 
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prototype  telepresence  systems  with  applications  in  minimally  invasive  surgery  (MIS), 
microsurgery,  and  remote  surgery. 

To  address  the  need  for  remote,  far-forward  battlefield  trauma  care,  the  MEDFAST 
telepresence  system  needs  responsive  telemanipulators  that  allow  the  surgeon  to  remotely 
perform  the  full  spectrum  of  surgical  maneuvers — ^for  example,  cutting,  suturing, 
dissecting — ^with  a  high  degree  of  dexterity  and  effectiveness  comparable  to  hands-on  skills. 
Similar  features  are  required  for  an  effective  surgical  simulation  and  training  system.  Systems 
providing  the  required  dexterity,  speed,  and  delicate  force  feedback  have  not  been  previously 
developed.  Only  SRI’s  telepresence  technology  provides  the  necessary  manipulation,  combined 
with  integrated  stereographic  and  audio  feedback,  that  makes  its  use  natural  and  effective. 

The  SRI  telepresence  technology  is  an  entirely  new  method  of  remote  manipulation,  with 
important  potential  applications  in  MIS,  remote  surgery,  and  surgical  training.  In  1996  SRI 
licensed  its  telepresence  technology  to  Intuitive  Surgical  Corporation,  a  venture-funded  start-up 
company  that  is  currently  focused  on  applying  the  technology  to  minimally  invasive  cardiac 
surgery.  Intuitive  has  completed  over  150  cardiac  surgeries  in  Evurope,  using  its  latest  telesurgery 
system,  and  the  U.S.  Food  and  Drug  Administration  (FDA)  recently  approved  Intuitive’s  device 
for  sale  in  the  United  States. 
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2.  PROJECT  ACCOMPLISHMENTS 


The  accomplishments  for  each  of  the  three  projects  are  described  below,  and  are 
discussed  in  reference  to  the  corresponding  approved  Statement  of  Work. 

2.1  DEVELOPMENT  OF  AN  ADVANCED  TELEPRESENCE  SURGERY  SYSTEM 

The  telepresence  surgery  system  developed  under  this  project  is  shown  in  Figure  1,  and  is 
seen  to  consist  of  two  modules:  an  RSU,  where  the  remote  surgery  actually  takes  place,  and  a 
TSW  with  a  strikingly  realistic  virtual  workspace  where  the  surgery  appears  to  take  place.  This 
system  incorporates  many  advanced  technologies  developed  at  SRI,  including  highly  responsive, 
force-reflecting  master/slave  manipulators  with  6  degrees-of-freedom  (DOF),  a  3D  video  display 
system  for  viewing  the  surgical  field,  and  stereophonic  sound.  In  effect,  the  operator  grasps 
surgical  instruments,  reaches  into  what  appears  to  be  the  actual  workspace,  and  carries  out 
complex  tasks  with  quick,  sure  motions.  Operating  surgeons  feel  as  though  they  are  performing 
tasks  right  before  their  own  eyes.  The  3D  image  is  created  by  a  stereographic  video  system, 
viewed  with  a  mirror.  The  surgeon  looks  downward,  “through”  the  mirror,  to  see  the  surgical  site 
below  it,  just  where  he  or  she  is  reaching  with  the  hand  controls.  The  left  and  right  control 
handles  are  mounted  on  light,  well-balanced,  force-reflecting  servo  manipulators — ^the  masters. 
Identical  slave  servo  manipulators  at  the  operating  table  support  the  actual  surgical 
instrument — scalpel,  forceps,  and  so  forth — ^the  tip  of  which  is  seen  in  the  image,  as  if  it  were 
actually  attached  to  the  handle.  When  the  tip  touches  the  tissue,  the  resistance  is  felt  through  the 
handle.  The  overall  effect  is  very  compelling. 

In  accordance  with  our  Statement  of  Work  for  this  project,  SRI  organized  the  MEDFAST 
development  effort  into  13  major  tasks.  Tasks  completed  during  Years  1  and  2  are  included 
below  for  completeness.  However,  their  status  is  imchanged  from  the  most  recent  annual  report. 

2.1.1  Task  1 :  Mission  Definition  (completed  in  Year  1) 

We  consulted  with  combat  and  trauma  surgeons,  combat  tacticians,  collaborating  contractors, 
and  military  telecommunications  specialists  to  define  the  surgeries  to  be  imdertaken  and  the 
conditions  and  constraints  xmder  which  the  system  must  operate.  For  further  information 
regarding  the  accomplishments  for  Task  1,  refer  to  the  project’s  annual  report  for  1996. 
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Telepresence  Surgeon’s  Workstation  (TSW) 


Remote  Surgery  Unit  (RSU) 


Figure  1 .  Advanced  Telepresence  Surgery  System 


2.1 .2  Task  2:  System  Specification  (completed  in  Year  2) 

A  simplified  block  diagram  of  the  overall  system  is  shown  in  Figure  2.  Performance, 
physical,  and  electrical  specifications  were  established  for  the  master  and  slave  telemanipulators, 
the  surgeon’s  video  display,  and  the  communication  link  between  the  TSW  and  RSU.  These 
specifications,  which  formed  the  basis  for  the  overall  MEDFAST  system  design,  are  summarized 
below. 

•  Manipulator  (left  and  right  arms) 

-  Degrees  of  freedom:  6  plus  instrument  actuation 

-  Motion  range:  12-in.  (300-mm)  cube 

-  Rotational  motion:  360°  on  tool  roll  and  yaw  axis,  150°  pitch  axis 
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Telepresence  Surgeon's 

Remote  Surgery 

Workstation  (TSW) 

Unit  (RSU) 

•  Two  6-DOF  Hand  Controls  with 
Interchangeable  Handles 

•  Positionable  Pod 

•  Two  6-DOF  Manipulators  with 

•  Stereographic  Display 

Interchangeable  Instruments 

•  Panoramic  Scene  Display 

Digital 

Microwave 

•  Stereographic  Video 

Cameras 

•  Auxiliary  Controls 

Communication 

Link 

•  Three  Scene  Cameras 

•  Computer  Workstation 

•  Servo  Controllers 

•  Computer  Workstation 

•  Servo  Controllers 

•  Stereo  Video  Processor 

•  Video  Codecs 

•  Video  Codecs 

•  Audio  Equipment 

•  Audio  Equipment 

H 

Figure  2.  MEDFAST  system  block  diagram.  The  complete  MEDFAST  system  consists  of  the 
telepresence  surgeon’s  workstation,  the  communication  link,  the  remote  surgery  unit,  and  electronics 
modules  associated  with  the  TSW  and  RSU. 


-  Force  applied:  3  Ibf  (13  N) 

-  Servo  bandwidth:  10  Hz 

-  Position  resolution:  0.004  in.  (0.1  nun) 

-  Force  resolution:  0.1  oz  (0.028  N) 

-  Inertia  at  handgrip:  <100  grams 

•  Control  handles  for  master  manipulators,  interchangeable  for  surgeon’s  use  with 
either  right  or  left  hand 

-  Hemostat  handle 

-  Forceps  (pickup)  handle 

•  Slave  instrument  interface 

-  Instruments  to  be  easily  changeable  automatically  and  by  hand 

-  Instruments  to  be  sterilizable  or  disposable 

•  Jaw  force  and  range  of  motion  for  articulated  instruments 

-  Hemostat:  7  Ibf  (opening  and  closing),  40° 
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-  Needle  driver:  45  Ibf  (closing  force),  15° 

-  Scissors:  7  Ibf  (closing),  15° 

-  Forceps:  2  Ibf  (closing),  0.5  in.  opening 

•  Stereo  vision  system 

-  Display  resolution:  640  x  480  pixels,  minimum 

-  Camera  resolution:  700  horizontal  by  494  vertical  lines 

-  Stereo  display  type:  120  Hz  full-frame  with  active  or  passive  glasses 

•  Communication  link 

-  Radio  type:  digital  microwave,  2-8  GHz  (line  of  site) 

-  Signal  format:  standard  DS3  and  DSl  digital  channels 

-  Servo  latency:  1  ms  (maximum) 

-  Video  latency:  50  ms  (maximum) 

-  Range:  5  km  (minimum) 

2.1.3  Task  3;  First  Preclinical  Demonstration  (completed  in  Year  1) 

Using  an  improved  version  of  the  SRI  telepresence  surgery  demonstration  system,  we 
have  successfully  performed  a  series  of  remote  surgical  procedures  on  pigs.  These  first-ever 
remote  surgeries  included  treatment  of  a  variety  of  simulated  abdominal  and  vascular  injuries, 
and  were  performed  by  surgeons  experienced  in  trauma  management.  The  successful  results  of 
these  experiments  clearly  demonstrate  the  feasibility  of  telepresence  surgery  for  emergency 
stabilization  of  battlefield  casualties.  In  addition,  MEDFAST  project  engineers  closely  observed 
the  experimental  procedures  and  held  in-depth  discussions  with  the  surgeons  to  gain  important 
feedback  and  suggestions  concerning  system  performance  and  features.  The  results  of  these 
experiments  and  discussions  are  serving  as  an  important  guide  in  the  development  of  the 
advanced  surgery  system. 

Operations  were  performed  on  4-  to  6-month-old  female  Hampshire  farm  pigs,  weighing 
60  to  80  pounds.  The  animals  were  housed  in  the  AAALAC-accredited  animal  care  facility 
(ACF)  at  UCSF,  and  used  under  an  approval  by  the  UCSF  and  SRI  Animal  Care  and  Use 
Committees.  The  animals  were  fasted,  except  for  water,  for  12  hours  prior  to  surgery.  They  were 
premedicated  in  the  ACF  with  atropine  (0.05  mg/kg),  ketimine  (20  mg/kg),  and  xylazine 
(2  mg/kg),  and  then  transported  to  the  animal  operating  room  and  anesthetized  with  sodium 
pentobarbital  (15-25  mg/kg  with  p.r.n.  supplements).  These  were  acute  experiments.  At  the 
conclusion  of  the  operations,  the  animals  were  euthanized  with  ketimine  (20  mg/kg)  and  xylazine 
(2  mg/kg)  followed  by  pentobarbital  (150  mg/kg,  IV). 
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All  procedures  attempted  were  completed  suecessfully,  including 

•  Cholecystectomy 

•  Nephrectomy 

•  Repair  of  gastrotomy  (simulated  stomach  injmy) 

•  Repair  of  enterotomy  (simulated  intestinal  injury)  by  primary  suture  and  by 
anastomosis 

•  Control  of  liver  laceration 

•  Repair  of  femoral  artery  laceration 

•  Aortic  interposition  graft 

•  Repair  of  ureteral  injury  by  stent  placement  and  primary  anastomosis 

•  Repair  of  bladder  laceration 

2.1.4  Task  4:  Modeling  and  Mock-up  (completed  in  Year  2) 

As  part  of  the  system  development  process,  we  used  computer  graphics  to  model  and 
evaluate  designs  for  the  TSW  and  RSU,  and  we  also  constructed  full-scale  wood  and  xmi-strut 
mock-ups  of  the  workstation,  operating  table,  and  surgical  pod.  These  models  and  mock-ups 
have  enabled  surgeons  and  members  of  the  design  team  to  interact  with  the  designs  early  in  the 
development  process,  and  feedback  from  these  interactions  has  played  an  important  role  in 
system  development.  For  example,  before  the  optimized  layouts  for  the  TSW  and  RSU  were 
defined,  the  mock-ups  were  used  to  experimentally  evaluate  a  series  of  alternative  moimting 
configurations  for  the  master  and  slave  manipulators.  The  mock-ups  and  models  also  played  an 
important  role  in  determining  the  seating  geometry  and  location  of  the  virtual  workspace  with 
respect  to  the  surgeon. 

2.1.5  Task  5:  Communication  Link  Development  (completed) 

As  shovra  in  the  block  diagram  of  Figure  3,  we  developed  a  two-way  communication  link 
that  provides  a  wireless  connection  between  the  TSW  and  the  MEDFAST.  All  subsystems  of  the 
communication  link  for  data  compression,  carrier  modulation,  microwave  transmission,  and  the 
inverse  operations  were  integrated,  tested,  and  debugged,  and  the  completed  communication  link 
was  successfully  used  in  the  vdreless  field  demonstration  described  Task  12.  The  microwave 
radio  imits  were  obtained  on  a  loan  basis  from  another  DARPA-sponsored  project  at  SRI.  All 
other  system  components  have  been  purchased  specifically  for  the  MEDFAST  system. 
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Surqeon's  Base  Unit  at  Hospital  '  Remote  Surgery  Unitin  Vehicb 
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2.1 .5.1  Radio  Subsystem 

To  implement  the  radio  connection  between  the  TSW  and  the  RSU,  we  selected  high- 
performance  digital  microwave  radio  units  (model  MDR-4208e)  manufactured  by  Alcatel 
Network  Systems.  The  MDR  4208e  is  a  well-proven  design  and  has  a  transmission  capacity  of 
two  DS3  channels  and  two  DSl  side  channels.  The  DS3  channels  (45  Mb/s)  were  dedicated  to 
the  stereo  video  for  the  surgeon’s  view,  while  the  DSl  channels  (1.54  Mb/s)  carry  servo  data  and 
a  multiplexed  data  stream  including  auxiliary  video,  peripheral  equipment  control  signals,  and 
information  on  life  support  for  trauma  and  transport  (LSTAT). 

To  verify  the  performance  of  the  selected  radios  and  the  overall  communication  link,  we 
carried  out  preliminary  experiments  in  December  1995  at  SRI’s  radio  field  test  site  in  Los 
Banos,  California.  A  major  goal  of  the  field  testing  was  to  evaluate  the  feasibility  of  using  a 
small,  low-gain  antenna  at  the  MEDFAST  vehicle  (in  combination  vAth  a  relatively  larger  dish  at 
the  “base”  site),  which  would  have  the  advantages  of  better  ruggedness  and  lower  visible  and 
radar  cross-sections.  In  the  tests,  die  mobile  and  base  antennas  were  mounted  on  towers  located 
6,000  feet  apart.  The  mobile  antenna  was  quite  small,  approximately  6  inches  in  diameter,  and 
was  supported  on  a  16-foot  mast.  The  base  antenna  was  6  feet  in  diameter  and  was  supported  on 
a  24-foot  mast.  Using  this  configuration,  the  digital  communication  link  provided  highly  reliable 
transmission  of  both  video  and  servo  data  in  both  clear  and  rainy  weather,  confirming  the 
effectiveness  of  both  the  antenna  configuration  and  the  MDR  4208  radio  units. 

2.1 .5.2  Video  and  Audio  Subsystems 

The  RSU  includes  video  cameras  for  the  surgeon’s  stereo  and  scene  views,  and  their 
output  video  streams  must  be  digitized  (via  codecs)  prior  to  radio  transmission.  Because  the 
latency  and  resolution  of  the  surgeon’s  3D  view  are  critical  to  system  performance,  SRI 
imdertook  a  careful  review  of  available  video  codecs.  Our  laboratory  testing  of  the  codecs 
included  comparative  and  absolute  resolution  measurements  using  calibrated  test  charts,  and 
latency  tests  using  switched  diode  light  sources  and  digital  timing  equipment.  We  evaluated 
several  high-performance  codec  models  from  Northern  Telecom  and  Alcatel  Network  Systems, 
and  found  that  the  Alcatel  model  1745  VC  provided  the  best  performance  in  terms  of  our  low- 
latency,  high-resolution  requirements,  in  part  because  it  performs  minimal  signal  compression. 
We  therefore  selected  the  1745  VC  codec  for  processing  the  3D  video.  The  1745  VC  also 
includes  a  pair  of  high-fidelity  audio  subchannels  that  will  provide  stereo  sound  feedback  to  the 
surgeon. 

Latency  was  less  of  an  issue  for  the  surgeon’s  auxiliary  scene  view,  and  so  we  chose  to 
compress  the  scene  video  (via  a  PictureTel  PCS- 100  unit)  to  fractional  DSl  bandwidth.  The 
PCS- 100  units  provide  duplex  transmission  with  good  image  and  sound  quality,  and  their 
fractional  DSl  output  can  be  multiplexed  onto  the  radio’s  available  DSl  bandwidth  along  with 
other  data. 

We  originally  proposed  to  employ  a  video  subsystem  based  on  SPARCstation  computers 
using  JPEG  or  MPEG  video  compression  cards  and  ATM  encoding;  hence,  we  began  Task  5  by 
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developing  and  testing  a  prototype  system  of  this  type  in  SRI’s  Telecommunication  Technology 
Laboratory.  However,  test  results  with  the  state-of-the-art  prototype  system  quickly  revealed  that 
the  combined  processes  of  image  compression  and  ATM  encoding  resulted  in  video  latencies  of 
approximately  200  ms,  well  beyond  our  30-ms  requirement.  We  considered  developing  custom 
software  and  hardware  to  speed  the  compression  and  encoding  processes,  but  concluded  that 
such  an  effort  was  beyond  the  scope  of  the  current  project.  Accordingly,  we  have  chosen  to  defer 
the  implementation  of  MPEG  and  ATM  technologies  until  adequate  systems  become 
commercially  available. 

2.1 .5.3  Servo  Communication  Subsystem 

Control  data  were  passed  between  the  TSW  and  RSU  servo  controllers  via  one  of  the 
DSl  radio  channels.  To  maximize  servo  performance  we  implemented  higher-resolution  (laser) 
position  encoders  on  the  new  master  and  slave  manipulators,  resulting  in  a  higher  servo  data  rate 
compared  with  our  original  system.  In  the  absence  of  compression  techniques,  this  position  data 
stream,  in  combination  with  force  sensor  data,  was  in  excess  of  the  1 .54  Mb/s  available  in  the  T1 
channel.  Therefore,  in  order  to  take  full  advantage  of  the  laser  encoders,  we  developed  a  lossless 
compression  algorithm  for  the  servo  data  stream  based  on  the  use  of  incremental  position  data  in 
combination  with  periodic  transmission  of  absolute  information  for  each  manipulator  arm. 

The  servo  controller  cards  reside  in  Pentium-II  personal  computer  (PC)  hosts  along  with 
T1  serial  port  cards.  To  implement  the  servo  communication  link,  we  developed  high-speed 
interface  software  for  the  host  PC  that  transfers  data  between  the  controllers  and  T1  ports  and 
continuously  monitors  data  integrity.  We  successfully  demonstrated  the  servo  communication 
subsystem  as  part  of  Task  12. 

2.1.6  Task  6:  Integration  of  Current  Surgery  System  with  MEDFAST  (completed  in  Year  1) 

The  existing  5 -DOF  demonstration  system — augmented  for  two-handed  operation  with 
interchangeable  surgical  instruments — ^was  integrated  with  the  MEDFAST  vehicle  in 
cooperation  with  Foster  Miller,  Inc.  The  integrated  mobile  system,  shown  in  Figure  4,  was 
successfully  demonstrated  (with  a  wired  communication  link)  at  the  1994  AUSA  Show,  the  1995 
National  Forum,  and  the  Pentagon  in  June  1995. 

In  addition  to  integration  with  the  MEDFAST,  the  telepresence  surgery  demonstration 
system  underwent  several  major  improvements:  (1)  Several  additional  types  of  interchangeable 
instruments  were  designed  and  built,  including  pickup-type  forceps,  curved  dissectors,  and 
“diamond  dust”  micro  needle  holders.  (2)  We  have  improved  the  responsiveness  and  stiffhess  of 
the  manipulators,  making  them  more  sensitive  to  touch,  through  the  addition  of  a  limited  integral 
term  to  the  servo  controller.  (3)  High-resolution,  triple-CCD  cameras  with  zoom  lenses  were 
added  for  improved  stereo  vision,  and  motorized  zoom  lens  controls  were  designed  and 
implemented.  These  improvements  significantly  increased  system  performance,  helping  to 
enable  the  successful  animal  surgeries  that  were  completed  as  part  of  Task  3. 
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(a)  Prototype  MEDFAST  vehicle 
containing  remote  surgery  unit 


(b)  View  inside  MEDFAST  vehicle 
showing  remote  surgery  unit  and 
medic  assistant 


Figure  4.  Integrated  mobile  system 


2.1 .7  Task  7:  Proof-of-Concept  Demonstration  (superseded) 

The  Task  6  configuration  was  originally  proposed  to  be  demonstrated  with  a  wireless 
communication  link  as  part  of  an  Advanced  Technology  Demonstration  (ATD)  exercise. 
However,  with  the  completion  of  Task  12,  the  communication  link  has  already  been  successfully 
used  for  wireless  operation  of  the  new  6-DOF  manipulators  in  the  MEDFAST  vehicle,  making 
Task  7  redundant. 

2.1.8  Task  8:  Telepresence  Surgeon’s  Workstation  (TSW)  Development  (completed) 

A  new  TSW  was  developed,  with  two  6-DOF  (plus  grip)  hand  controls,  enhanced  video 
displays,  auxiliary  controls,  and  all-new,  ruggedized  electronics.  In  designing  the  TSW  we  made 
extensive  use  of  the  TSW  and  RSU  mock-ups  from  Task  4.  Using  the  mock-ups,  we  had 
surgeons  perform  simulated  surgeries,  including  suturing  organ  models,  to  evaluate  the  range  of 
motions  required  for  surgical  procedures  and  guide  our  designs  as  they  evolved.  In  completing 
the  TSW,  we  evaluated  its  functional  performance  along  with  the  rest  of  the  MEDFAST  system, 
using  the  comments  of  surgeons  and  other  system  users  to  revise  and  expand  the  TSW  design. 
Consulting  surgeons  had  experience  in  trauma,  general,  and  vascular  surgery,  as  well  as  other 
surgical  specialties. 
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2.1 .8.1  Subtask  8.1:  Master  Manipulator  Design 

The  new  manipulators  were  designed  in  accordance  with  the  performance  specification  of 
Task  2.  Major  features  of  the  manipulator  design  are  discussed  below.  A  more  detailed 
description  of  the  manipulators  is  in  the  1996  annual  report. 

Manipulator  Articulation:  The  manipulator  geometry  incorporates  a  series  kinematic 
design  with  shoulder  (2  DOF),  elbow  (1  DOF),  and  wrist  (3  DOF)  and  integral  handgrip  actuator. 
All  seven  axes  are  driven  by  servo  motors  mounted  at  the  shoulder  so  as  to  maintain  a  low  inertia 
at  the  handgrip.  The  shoulder  and  elbow  are  driven  through  gears  and  push-rods  while  the  wrist 
and  gripper  are  driven  through  lightweight,  flexible  steel  cables. 

Wrist:  The  wrist  incorporates  bevel  gears  and  provides  roll,  pitch,  and  yaw  motions.  It 
also  incorporates  a  two-way  drive  to  positively  activate  the  tools  in  both  opening  and  closing. 

A  novel  external  wrist  yoke  provides  for  attachment  of  the  slave  tools  and  the  master  handgrips. 

Grip  Activation:  Each  handgrip  operates  a  push-rod  at  its  center  that  is  coupled  through 
linkages  and  cables  to  a  drive  motor  mounted  at  the  shoulder.  The  handgrips  employ  a  special 
balanced  mechanism  that  does  not  impart  any  net  forces  to  the  wrist  joint  during  instrument 
actuation,  and  thus  allows  wrist-mounted  force-torque  sensors  to  be  successfully  employed.  Such 
sensors  would  provide  enhanced  force  feedback  from  the  slave  instruments  and  would  also 
enable  “power  steering”  of  the  master  manipulators  to  minimize  friction  and  inertial  loads  on  the 
operator. 

Handgrip  Design:  The  new  design  allows  fi'ee  positioning  and  access  throughout  the 
workspace,  and  employs  control  handles  with  a  pistol-grip  design. 

2.1 .8.2  Subtask  8.2:  TSW  Console 

Human  Factors:  The  surgeon’s  console  was  designed  to  accommodate  surgeons  with  a 
wide  range  of  physical  statures.  The  design  comfortably  seats  men  and  women  ranging  from  the 
5th  to  the  95th  percentile  in  height,  and  enables  effective  telepresence  for  all  of  them.  Figure  5, 
based  on  data  from  Woodson  and  Conover  [1964],  illustrates  the  latitude  of  adjustment 
necessary.  As  shown  in  Figure  1,  the  surgeon  sits  on  a  chair  that  can  be  adjusted  to  the 
appropriate  height  so  that  his  or  her  eyes  are  in  the  optimal  location  to  view  the  display. 

TSW  Vision  System:  Both  a  heads-up  display  and  a  stereographic  display  are  included  in 
the  surgeon’s  workstation.  The  heads-up  display  is  located  at  the  rear  of  the  TSW  console,  and  is 
positioned  at  approximately  the  same  viewing  distance  as  the  stereo  display  to  minimize  the  need 
for  visual  accommodation  between  the  two  views.  The  stereographic  display  is  viewed  through  a 
mirror,  as  in  our  previous  telepresence  demonstration  system,  so  that  the  surgical  field  appears  to 
be  located  at  the  surgeon’s  fingertips.  Effective  visualization  of  the  surgical  field  is  critical  to  the 
surgeon’s  performance,  so  we  carefully  reviewed  the  state  of  the  art  in  3D  vision  systems  to 
identify  the  best  technology  for  the  TSW  vision  system.  After  a  complete  evaluation  of  display 
technologies,  we  selected  and  purchased  two  key  components  for  use  in  the  TSW:  (1)  a  five- 
segment  liquid-crystal  stereo  shutter  from  NuVision  Technologies  of  Beaverton,  Oregon,  and 
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Figure  5.  Variation  in  human  dimensions 


(2)  a  24-bit,  40  MHz  RGB  stereo  processor  system  from  Imaging  Technology  Corporation  of 
Bedford,  Massachusetts. 

In  selecting  the  stereo  shutter  we  evaluated  a  number  of  competing  technologies, 
including  light- valve  (active)  shuttered  glasses,  multisegment  120  Hz  liquid-crystal  shutters  with 
passive  glasses,  and  binocular  head-mounted  viewing  systems  with  both  liquid-crystal  and 
cathode-ray  tube  displays.  Objective  performance  was  measured  using  precision  photometry 
equipment  and  color  resolution  charts,  and  subjective  data  was  gathered  from  user  interviews.  In 
terms  of  image  resolution,  brightness,  and  contrast,  as  well  as  crosstalk  and  user  comfort,  the 
recently  developed  5-segment  polarizing  screen  from  NuVision  (in  combination  with  passive 
glasses  and  a  high-resolution  120  Hz  monitor)  was  found  to  be  the  most  effective  stereo  shutter 
technology  for  use  in  the  TSW. 

To  select  the  stereo  processor  for  the  new  TSW,  we  evaluated  the  color  performance, 
spatial  resolution,  and  graphical-overlay  capabilities  of  several  competing  systems,  including 
units  from  RGB  Spectrum  (Alameda,  California),  MATROX  Electronics  (Dorval,  Quebec), 
DataCube,  Inc.  (Danvers,  Massachusetts),  and  Imaging  Technologies.  A  system  from  Imaging 
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Technologies  (based  on  its  model  MVC  150/40  pipeline  processor)  offers  the  best  performance 
at  a  competitive  price.  The  MVC  150/40  system  provides  the  TSW  with  24-bit  “true”  color  and  a 
spatial  resolution  of  756  X  480  pixels. 

2.1 .8.3  Subtask  8.3:  Servo  Controller 

New  servo  electronics  and  system  software  were  developed  to  control  the  6-DOF  master 
and  slave  manipulators,  and  we  also  developed  servo  algorithms  that  enable  enhanced  tactile 
feedback  and  dynamic  response  for  the  manipulators.  To  design  the  new  algorithms  we  used 
both  numerical  and  analytical  modeling  techniques,  in  combination  with  experimental  testing  on 
a  single-axis  servo  test  stand.  Refer  to  the  1996  annual  report  for  a  complete  description  of  the 
many  accomplishments  of  Subtask  8.3. 

2.1 .9  Task  9:  Advanced  Remote  Surgery  Unit  (RSU)  Development  (completed) 

The  new  RSU  is  shown  in  Figure  1,  and  incorporates  a  medic-positionable  pod  with  two 
6-DOF  (plus  grip)  manipulators,  ruggedized  electronics,  and  an  array  of  interchangeable  surgical 
instruments.  The  RSU  of  Task  9  consists  of  many  of  the  same  components  as  the  TSW  of 
Task  8,  including  the  ruggedized  electronics  and  the  design  for  the  manipulators,  which  have 
been  previously  described.  Additional  components  for  the  RSU  are  described  below. 

2.1 .9.1  Slave  Manipulator  Design 

The  RSU  manipulators  are  essentially  identical  to  the  TSW  master  manipulators  of 
Task  8,  except  that  they  support  a  range  of  interchangeable  instruments  rather  than  control 
handles.  We  developed  a  unique  and  effective  mounting  mechanism  that  allows  these 
instruments  to  be  attached  to  the  manipulators  with  a  simple  twist-lock  action,  so  that  instrument 
interchange  can  be  easily  accomplished  remotely  (by  the  surgeon)  or  directly  by  hand  (by  the 
medic). 

Tool  Actuation:  Each  articulated  tool  is  operated  by  a  push  rod  at  its  center.  Pulling  the 
rod  closes  the  jaws  and  pushing  the  rod  opens  them.  Opening  and  closing  are  accomplished  by  a 
balanced  spreading  mechanism  that  does  not  impart  any  forces  to  the  force  sensor. 

Tool  Changer:  Different  tools  are  attached  to  the  manipulator  by  using  a  taper 
(nonlocking)  with  antirotation  pin  and  a  spring-loaded  collar  to  retain  the  tool  in  place.  The  tool 
mounting  interface  includes  a  unique  coupling  mechanism  that  supports  differential  force 
transmission  to  the  tool’s  actuator  pushrod,  preventing  the  transmission  of  spurious  loads  to  the 
wrist  force  and  torque  sensor.  In  operation,  the  collar  can  be  manually  slid  back  and  the  tool 
twisted/pulled  out,  or  the  manipulator  can  place  the  tool  in  a  passive  tool-changing  rack  and  then 
twist/pull  to  eject  it.  The  twist/pull  operation  also  automatically  disengages  the  differential-force 
coupling  to  the  tool-actuation  pushrod. 

RSU  Vision  System:  The  RSU  provides  the  surgeon  with  a  “heads  up”  scene  video  in 
addition  to  a  high-resolution  stereo  view  that  includes  remotely  controlled,  motorized  zoom 


16 

SRI  PROPRIETARY 


SRI  PROPRIETARY 


lenses  and  focus  control.  Because  the  stereo  view  is  critical  to  system  performance,  we 
performed  a  laboratory  evaluation  of  the  leading  high-performance  video  cameras  to  select  the 
most  effective  model.  Our  test  results  demonstrated  that  the  Panasonic  GP-US502  provided  the 
highest  resolution  (700  lines)  and  lowest  level  of  fnnging  artifacts.  Accordingly,  we  used  a  pair 
of  these  3-CCD  cameras,  fitted  with  motorized  zoom  lenses,  to  generate  the  3D  view. 

2.1.10  Task  10:  Integration  of  Advanced  Surgery  System  with  MEDFAST  vehicle  (completed  in 
Year  2) 

In  September  1996  the  new  RSU,  including  6-DOF  slave  manipulators,  stereo  vision 
cameras  with  motorized  zoom,  stereo  audio  microphones,  and  medic/surgeon  communication 
chaimel,  was  installed  in  an  M577  Armored  Ambulance  for  use  in  the  Task  12  field 
demonstration.  The  M577  was  outfitted  (by  Foster  Miller,  Inc.)  with  telesurgery  support 
equipment,  including  an  overhead  xyz  positioning  gantry  to  support  the  slave  manipulators  and 
stereo  cameras,  lighting,  storage  cabinets,  and  a  retractable  surgical  table  for  use  in 
demonstration  surgeries. 

2.1 .1 1  Task  1 1 :  Second  Preclinical  Demonstration  (completed  at  USUHS) 

In  July  1997,  SRI  delivered  a  complete  advanced  telepresence  surgery  system  to  USUHS 
as  part  of  the  third  project  carried  out  under  the  grant.  Using  this  system.  Dr.  Christoph 
Kaufmann  and  others  at  USUHS  have  successfully  carried  out  a  set  of  selected  surgical 
procedures  on  pigs  as  part  of  a  training  program  for  medical  students  at  the  university. 

2.1.12  Task  12:  Field  Demonstration  of  Advanced  MEDFAST  System  (completed) 

On  7  and  8  October  1996,  the  new  6-DOF  Advanced  Telesurgery  System  was  used  to 
demonstrate  wireless  telesiurgery  on  the  SRI  Menlo  Park  campus.  The  TSW  was  located  in  our 
research  laboratory  and  was  coupled  via  the  microwave  radio  link  to  the  RSU  located  outside  in 
the  M577  vehicle.  The  RSU  and  M577  are  shown  in  Figures  6a  and  6b.  The  system  was 
evaluated  by  a  number  of  visiting  surgeons,  including  Sam  Wells  (President  of  the  American 
College  of  Surgeons),  all  of  whom  commented  on  the  effectiveness  and  ease  of  operation 
afforded  by  the  new  6-DOF  manipulators  with  their  1-ft^  workspace  and  3-pound  force  capacity. 
During  the  demonstration,  the  system  was  used  to  perform  suturing  and  complete  a  dexterity  test 
based  on  3D  positioning  and  placement  tasks. 

2.1.13  Task  13:  Documentation  (completed) 

In  accordance  with  the  grant  agreement  for  this  project,  SRI  has  submitted  annual  project 
reports  and  is  submitting  this  final  project  report.  In  addition  to  these  required  reports,  SRI 
submitted  a  videotape  entitled  “Telepresence  for  Far  Forward  Surgical  Intervention  on  the 
Battlefield”  on  30  November  1994,  as  well  as  a  written  project  status  report  on  10  July  1995. 
During  the  system  development,  complete  mechanical  and  electronic  drawings  and  parts  lists. 
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(a)  Interior  view  showing  the  new  RSU  with  6-DOF  slave 
manipulators  (left  side).  The  seated  medic  is  shown 
assisting  the  remote  surgeon. 


(b)  Exterior  view  showing  the  roof-mounted  microwave  antenna 
used  for  wireless  control  of  the  RSU  mounted  inside. 


Figure  6.  New  RSU  in  M577  with  microwave  link 

software,  instruction  manuals  for  purchased  subsystems,  and  system  operating  instructions  were 
prepared  and  compiled  into  manuals. 
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2.2  TELEPRESENCE  SURGERY  SIMULATION 

In  accordance  with  our  Statement  of  Work  for  this  project,  SRI  has  organized  the  Surgical 
Simulation  effort  into  three  major  tasks,  with  the  goal  of  incorporating  physically  based,  virtual 
surgical  environments  into  the  SRI  Telepresence  Surgery  System.  Our  aim  was  to  develop  a 
platform  that  could  be  used  to  demonstrate  and  evaluate  surgical  simulation  and  training  models 
that  were  being  developed  by  other  groups,  including  the  models  developed  by 
MusculoGraphics,  with  support  from  DARPA’s  Biomedical  Technology  Program. 

2.2.1  Taski:  Development  of  Basic  Platform  (completed) 

We  first  developed  a  basic  platform  for  surgical  simulation  incorporating  simple 
simulation  models  and  a  single  control  manipulator.  The  platform  consisted  of  a  PC-based  servo 
control  module,  a  single  6-DOF  control  manipulator,  and  an  (SRI-owned)  SGI  graphics 
workstation  for  running  the  surgical  model.  We  used  this  platform  for  initial  design  and  testing 
of  our  control  algorithms  and  for  developing  the  human  interface.  Specific  accomplishments  for 
this  task  include  the  following. 

Control  Interface:  Because  of  the  enormous  number  of  calculations  required  to  simulate 
deformable  tissues,  visual  display  update  rates  with  state-of-the-art  graphics  workstations  are 
typically  limited  to  about  20  Hz.  If  the  graphics  engine  were  used  to  calculate  haptic  (force- 
feedback)  signals  for  the  simulation  environment,  this  same  low  update  rate  would  be  imposed 
on  the  force-feedback  and  control  loops — ^with  imdesirable  effects  on  system  performance.  In  the 
case  of  force  feedback,  the  low  update  rate  would  cause  an  annoying  20  Hz  buzzing  vibration  in 
the  control  manipulator.  The  effect  on  the  servo  control  loop  would  be  to  limit  its  bandwidth, 
causing  the  surgical  instruments  to  feel  spongy  or  rubbery. 

To  avoid  these  pitfalls  we  developed  a  system  in  which  the  haptic  calculations  and 
graphics  processing  are  performed  on  separate  computers.  By  using  this  approach  we  are  able  to 
maintain  a  force  and  servo  update  rate  in  excess  of  1  kHz,  and  thus  avoid  buzzing  or  spongy 
behavior.  By  separating  the  visual  and  haptic  models  we  can  reduce  the  complexity  of  the  haptic 
model  as  needed  to  achieve  adequate  processing  speed,  while  maintaining  a  high-quality  visual 
rendering.  Since  humans  are  much  more  sensitive  to  visual  details  than  haptic  details,  this 
approach  yields  substantial  benefits. 

To  implement  this  system,  SRI  developed  a  serial  interface  between  the  Pentium  II  PC 
system  and  the  SGI  workstation  and  also  developed  software  to  transfer  positions  and  forces  to 
and  from  the  simulation  software.  For  positioning  the  simulated  tools,  we  developed  the 
kinematic  solution  for  the  manipulators,  and  for  force  feedback  we  developed  their  Jacobians. 
The  kinematic  solutions  and  Jacobians  are  modular  so  that  code  for  different  manipulators  can  be 
quickly  developed  as  needed. 
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Development  Hardware:  To  cany  out  our  development  work,  we  designed,  purehased, 
and  assembled  a  PC-based  servo  control  module  consisting  of  a  commercially  available  DSP 
servo  control  board  and  servo  amplifier  boards  packaged  in  a  small  electronics  cabinet. 

Simple  Models:  To  test  and  debug  our  Phase  I  platform,  we  developed  simple  simulation 
models  for  a  blunt  surgical  probe  that  moves  according  to  input  commands  following  the  master 
manipulator.  The  manipulator  kinematics  software  with  Jacobian  for  force  generation  were 
implemented  to  make  the  virtual  tools  move  according  to  manipulator  input  from  the  master.  In 
addition,  we  generated  a  simple  compliant  model — consisting  of  a  3-sided  “box”  with  compliant 
walls — ^for  testing  and  debugging  initial  system  operation. 

System  Integration  and  Evaluation:  We  integrated  the  SGI  workstation,  the  servo  PC 
computer,  and  the  master  manipulator  controllers  into  a  complete  and  functional  Phase  I 
simulation  system,  and  used  the  system  to  evaluate  our  design  approach  and  gather  data  to  guide 
the  design  of  the  final  Phase  II  simulation  platform. 

2.2.2  Task  2:  Integration  with  SRI  Telesurgery  Demonstration  System  (superseded) 

SRI  originally  proposed  to  develop  an  intermediate-stage  platform  using  SRI’s  original  4- 
DOF  telepresence  system  in  combination  with  SRI’s  Reality  Engine  image  processing  computer. 
At  the  conclusion  of  Task  1,  however,  our  progress  in  software  and  kinematic  algorithm 
development  was  ahead  of  schedule  and  so  we  were  able  to  move  directly  to  implementing  the 
full  6-DOF  final  system  in  Task  3. 

2.2.3  Task  3:  Upgrade  to  6  DOF  and  Integration  with  MEDFAST  (completed) 

Specific  accomplishments  for  Task  3  included  the  following. 

Implementation  with  6-DOF  Manipulators  of  MEDFAST  TSW:  We  expanded  the  Task  1 
system  to  accommodate  the  6-DOF  (plus  grip)  master  manipulators  of  the  newly  developed  SRI 
MEDFAST  system.  This  step  included  changing  the  Jacobians  and  kinematic  equations  to 
expand  both  position-out  and  force-in  calculations  from  3  axes  to  14  axes  of  motion. 

Implementation  of  Instrument  Models:  We  expanded  the  simulation  to  include  virtual 
surgical  instruments  (clamps  and  forceps)  like  those  we  developed  for  the  MEDFAST  system. 

Incorporation  of  Simulation  Models  and  Demonstration  of  System:  We  imported  the 
intravenous  (IV)  needle  insertion  trainer  and  the  leg  trauma  simulation  (LTS)  models  produced 
by  MusculoGraphics,  and  demonstrated  operation  of  these  models  on  the  SRI  platform.  With  the 
completion  of  this  step,  it  became  possible  to  use  the  TSW  to  perform  actual  remote  surgery  as 
well  as  practice  surgery  on  virtual  models.  In  fact,  the  surgeon  or  trainee  can  switch  seamlessly 
between  the  two  modes  with  a  few  simple  keystrokes. 

Instaliation  of  SRI  Simulation  Platform  at  USUHS:  To  provide  increased  access  to  the  SRI 
simulation  platform,  we  installed  the  platform  at  USUHS  where  it  will  form  part  of  the  new 
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Simulation  and  Readiness  Center  that  is  nearing  completion  there.  While  this  effort  was  beyond 
the  scope  of  the  proposed  work,  we  felt  that  it  was  important  to  make  the  system  more  widely 
available,  and  we  were  able  to  complete  the  installation  within  our  original  budget.  The 
installation  effort  involved  upgrading  the  MEDFAST  system  we  installed  there  in  July  1997,  and 
included  supplying  and  integrating  a  new  Pentium-II  host  computer  for  the  servo  system, 
installing  upgraded  software  and  cabling,  and  training  USUHS  staff  to  operate  and  maintain  the 
system. 

2.3  TESTBED  FOR  TELEPRESENCE  SURGERY  AT  USUHS 

The  scope  of  this  effort  was  to  provide  USUHS  with  a  complete  6-DOF  MEDFAST 
system  comprising  a  TSW,  RSU,  and  system  control  electronics.  SRI  fabricated  and  assembled 
the  complete  system,  installed  it  at  USUHS,  provided  technical  support  for  system  operation  and 
maintenance,  and  installed  system  upgrades  that  were  subsequently  developed  under  the  grant. 

As  proposed,  the  project  was  divided  into  four  tasks. 

2.3.1  Taski:  System  Fabrication  (completed) 

SRI  purchased,  fabricated,  and  assembled  all  the  components  required  for  the  complete, 
6-DOF  prototype  MEDFAST  system,  including  a  TSW,  RSU,  and  control  module.  The  TSW 
included  a  complete  3D  viewing  system,  telesurgical  instrument  control  handles,  ergonomic 
seating  for  the  surgeon,  and  scene  LCD  monitors.  The  RSU  included  high-resolution,  color  CCD 
cameras  for  stereo  viewing,  a  scene  CCD  camera,  manipulator  arms  with  manually 
interchangeable  surgical  instruments,  and  lighting.  Audio  linkage  between  the  TSW  and  RSU 
was  also  included.  The  assembled  system  is  identical  to  the  one  shown  in  Figure  1,  and  was  fully 
tested  and  debugged  at  SRI  facilities  in  Menlo  Park  before  shipment  to  USUHS  in  July  1997. 

2.3.2  Task  2:  Preparation  of  Facilities,  Shipment  and  instaiiation  of  System  (completed) 

In  conjunction  with  USUHS,  SRI  determined  the  modifications  required  to  the  USUHS 
physical  plant  to  accommodate  the  telepresence  surgery  system.  SRI  disassembled  the 
telepresence  surgery  system,  purchased  and  fabricated  padded  cases  and  transport  crates,  and 
packaged  and  shipped  the  system  fi'om  SRI  in  Menlo  Park  to  USUHS  in  Bethesda.  We  then 
unpacked  and  assembled  the  telepresence  surgery  system  at  USUHS,  and  tested  it  to  confirm  that 
all  subsystems  were  functioning  properly. 

2.3.3  Task  3:  System  Operation  and  Scientific  Testing  (compieted) 

A  scientific  testing  program  for  the  telepresence  system  was  designed  and  carried  out  by 
USUHS  staff  under  the  direction  of  Dr.  Kaufmann.  The  system  performed  very  reliably,  and 
only  minimal  technical  support  from  SRI  was  needed.  Dr.  Kaufmann  continues  to  use  the  system 
in  his  teaching  and  research  programs  at  USUHS. 
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2.3.4  Task  4:  System  Maintenance,  Support,  and  Upgrades  (completed) 

SRI  provided  three  major  upgrades  for  the  SRI  telepresenee  system  at  USUHS.  First,  we 
designed,  fabricated,  and  delivered  additional  surgical  instruments  for  the  system  as  requested  by 
Dr.  Kaufinann,  including  surgical  “pick-up”  forceps,  curved  forceps,  and  improved  needle 
holders.  In  the  next  upgrade,  we  installed  a  higher-resolution  (756  x  480  pixel)  24-bit  RGB  color 
stereo  processor  that  we  had  under  development  to  replace  the  existing  NTSC  stereo  processor. 
Our  final  upgrade  was  a  major  effort  that  enabled  the  TSW  to  be  used  as  an  interface  to  access 
computer-based  surgical  simulation  and  training  models.  This  upgrade  was  carried  out  as  an 
extension  of  the  Telepresence  Surgery  Simulation  project  as  described  in  Section  2.2 

2.4  KEY  RESEARCH  ACCOMPLISHMENTS 

•  In  April  1995  we  used  an  improved  version  of  the  SRI  telepresence  surgery 
demonstration  system  to  successfully  perform  a  series  of  remote  surgical  procedures  on 
pigs.  These  first-ever  remote  surgeries  included  treatment  of  a  variety  of  simulated 
abdominal  and  vascular  injuries,  and  were  performed  by  surgeons  experienced  in  trauma 
management.  The  successful  results  of  these  experiments  clearly  demonstrated  the 
feasibility  of  telepresence  surgery  for  emergency  stabilization  of  battlefield  casualties. 

All  procedures  attempted  were  completed  successfully. 

•  On  28  August  1996,  we  demonstrated  remote  telesurgery  operation  using  the  DARPA- 
funded  National  Transparent  Optical  Network  Consortium  (NTONC)  fiber-optic  lines  as 
the  commimication  link.  In  this  demonstration  the  4-DOF  SRI  system  was  operated  over 
a  300-km  multiple-wavelength  all-optical  network  (including  optical  switches  and 
amplifiers)  provided  by  the  NTONC  project  at  Lawrence  Livermore  National  Laboratory 
(Livermore,  California).  The  TSW  was  located  at  the  national  laboratory  in  Livermore, 
and  the  RSU  was  located  at  SRI  (Menlo  Park,  California).  Consortium  executives  as  well 
as  DARPA  program  manager  Bert  Hui  used  the  system  during  a  project  meeting  and 
were  impressed  with  the  effectiveness  of  the  telepresence  technology  and  its  ability  to 
operate  over  long  distances. 

•  On  7  and  8  October  1996,  we  demonstrated  wireless  telesurgery  with  the  new  6-DOF 
Advanced  Telesurgery  System.  The  TSW  was  located  in  our  research  laboratory  in 
Menlo  Park  and  was  coupled  via  a  microwave  radio  link  to  the  RSU  located  outside  in  an 
M577  armored  ambulance.  The  system  was  evaluated  by  a  number  of  visiting  surgeons, 
including  Sam  Wells  (President  of  the  American  College  of  Surgeons),  all  of  whom 
commented  on  the  effectiveness  and  ease  of  operation  afforded  by  the  new  6-DOF 
manipulators  with  their  1-tf  workspace  and  3-pound  force  capacity.  During  the 
demonstration,  the  system  was  used  to  perform  suturing  and  complete  a  dexterity  test 
based  on  3D  positioning  and  placement  tasks. 

•  In  July  1997,  SRI  delivered  a  complete,  6-DOF  prototype  MEDFAST  system  to  USUHS, 
including  a  TSW,  RSU,  and  control  module.  The  TSW  included  a  complete  3D  viewing 
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system,  telesurgical  instrument  control  handles,  ergonomic  seating  for  the  surgeon,  and 
scene  LCD  monitors.  The  RSU  included  high-resolution,  color  CCD  cameras  for  stereo 
viewing,  a  scene  CCD  camera,  manipulator  arms  with  interchangeable  surgical 
instruments,  and  lighting.  An  audio  communication  link  between  the  TSW  and  RSU  was 
also  included. 

•  In  Jime  1999,  we  installed,  at  USUHS,  a  Telepresence  Simulation  platform  that  enables 
surgeons  and  medical  students  to  interact  with  computer-based  surgical  simulation  and 
training  models  using  the  MEDFAST  TSW.  The  TSW  provides  advanced  haptic  and 
visual  interfaces,  with  force  and  torque  feedback  on  all  6  DOF  of  the  master  manipulators 
and  high-resolution  stereo  video.  The  MEDFAST  and  Telepresence  Simulation  platform 
are  expected  to  play  an  important  role  in  the  new  Simulation  and  Readiness  Center  that  is 
nearing  completion  at  USUHS. 

2.5  REPORTABLE  OUTCOMES 

In  addition  to  successfully  completing  the  projects  and  tasks  discussed  above,  the  work 
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3.  CONCLUSIONS 


In  successfully  completing  the  three  projects  covered  by  this  report  (Advanced 
Telepresence  Surgery  System,  Surgical  Simulation,  and  Telepresence  Testbed  for  USUHS),  SRI 
has  substantially  advanced  the  state  of  the  art  in  the  fields  of  telepresence  surgery  and  surgical 
simulation  and  training.  By  developing  and  demonstrating  the  MEDFAST  telesurgery  system, 
we  have  clearly  proven  the  technical  feasibility  of  performing  remote,  wireless  telesurgery  to 
provide  trauma  care  to  wovmded  personnel.  No  system  like  the  MEDFAST  existed  prior  to  this 
development  effort,  and  the  Telepresence  Testbed  at  USUHS  can  now  serve  as  a  technology 
center  that  will  build  on  our  work  and  define  the  next  steps  to  be  taken  toward  practical  field 
implementation.  In  the  area  of  Surgical  Simulation,  the  SRI  Surgical  Simulation  platform  we 
installed  at  USUHS  provides  a  level  of  realism  and  haptic  fidelity  that  is  unmatched  by  any  other 
system.  As  a  result,  the  USUHS  Testbed  can  also  serve  as  a  basis  from  which  to  specify  and 
design  the  next-generation  simulation  and  training  systems,  and  important  user  feedback  can  be 
provided  by  the  medical  students  and  surgical  staff  at  that  facility. 

With  the  ever-increasing  network  speed  and  computer  processing  power  that  becomes 
available  each  year,  the  telepresence  and  simulation  technologies  represented  by  the  SRI  systems 
will  rapidly  become  more  powerful  and  more  cost-effective  to  implement.  In  the  near  future, 
low-orbit  satellites  will  be  available  to  provide  high-bandwidth,  low-latency  communication 
links  for  wireless  telepresence  surgery.  For  simulation,  increased  processing  speed  and  memory 
capacity  will  enable  simulation  systems  that  provide  unprecedented  realism  in  surgical  planning 
based  on  software  models  with  patient-specific  anatomy. 

It  is  also  important  to  note  that  the  telepresence  technology  developed  by  SRI  under  this 
grant  can  potentially  be  of  significant  value  to  a  wide  range  of  military  customers.  The  key  areas 
of  technology  development  are 

•  Integration  of  a  range  of  advanced  control,  display,  and  robotic  technologies  to  create 
a  transparent,  yet  immersive,  interface  that  enables  a  human  to  effectively  carry  out 
remote  manipulation  tasks  and  can  also  serve  as  a  surgical  simulation  user  interface 
of  unparalleled  fidelity  and  realism 

•  Development  of  advanced  control  algorithms  to  provide  a  natural  haptic  interface  to 
the  remote  world 

•  Development  of  state-of-the-art  6-DOF  master  and  slave  manipulators 

•  A  unique  mapping  strategy  between  master  and  slave  to  enable  effective  human-in- 
the-loop  control 
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Several  branches  of  the  military  as  well  as  non-DOD  eustomers  stand  to  benefit  from 
these  technologies.  Some  of  the  key  customers  and  their  potential  applications  are 

•  U.S.  Army:  Telepresence  surgery  for  remote  combat  casualty  care  and  local 
treatment  of  individuals  (in  containment  facilities)  exposed  to  biological  or  chemical 
agents 

•  U.S.  Navy:  Medical  care  between  carriers  and  other  ships  in  fleet  without  the 
necessity  to  transport  medical  personnel 

•  Explosive  Ordnance  Disposal  (EOD):  Capability  for  EOD  technicians  to  remotely 
investigate  and  disarm  unexploded  ordinance,  terrorist  devices,  and  mines  without 
exposing  personnel  to  risk  of  injury  from  accidental  explosions 
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Vascular  applications  of  telepresence  surgery: 
Initial  feasibility  studies  in  swine 
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Paul  R.  Cordts,  MD,  and  Pl^p  S.  Green,  MS,  Beihesda,  Md.^,  Menlo  Parky  Calif.y 
and  Honoluluy  Hawaii 

Purpose:  Telepresence  sxxrgery  is  a  novel  technology  that  will  allow  procedures  to  be 
performed  on  a  patient  at  locations  that  are  physically  remote  from  the  operating  surgeon. 

This  new  method  provides  the  sensory  illusion  that  the  surgeon’s  hands  are  in  direct 
contact  with  the  patient.  We  studied  the  feasibility  of  the  use  of  telepresence  surgery  to 
perform  basic  operations  in  vascular  surgery,  including  tissue  dissection,  vessel  manipu¬ 
lation,  and  suturing. 

Methods:  A  prototype  telepresence  surgery  system  with  bimanual  force-reflective  manipu¬ 
lators,  interchangeable  surgical  instruments,  and  stereoscopic  video  input  was  used. 
Arteriotomies  created  ex  vivo  in  segments  of  bovine  aortae  or  in  vivo  in  femoral  arteries 
of  anesthetized  swine  were  closed  with  telepresence  siu-gery  or  by  conventional  techniques. 

Time  required,  technical  quality  (patency,  integrity  of  suture  line),  and  subjective  difficulty 
were  compared  for  the  two  methods. 

Results:  All  attempted  procedures  were  successfully  completed  with  telepresence  surgery. 
Arteriotomy  closures  were  completed  in  192  ±  24  sec  with  conventional  techniques  and 
483  ±118  sec  with  telepresence  surgery,  but  the  precision  attained  with  telepresence 
surgery  was  equal  to  that  of  conventional  techniques.  Telepresence  surgery  was  described 
as  intuitive  and  natural  by  the  surgeons  who  used  the  system. 

Conclusions:  Blood-vessel  manipulation  and  suturing  with  telepresence  siurgery  are 
feasible.  Further  instrument  development  (to  increase  degrees  of  freedom)  is  required  to 
achieve  operating  times  comparable  to  conventional  open  surgery,  but  the  system  has  great 
potential  to  extend  the  expertise  of  vascular  smgeons  to  locations  where  specialty  care  is 
currently  unavailable.  (J  Vasc  Surg  1996;23:281-7.) 


Surgery  has  always  required  that  the  hands  of  a 
surgeon  be  placed  directly  in  a  wound.  The  rapid 
emergence  of  minimally  invasive  surgery,  however, 
has  conclusively  proven  that  for  many  procedures 
indirect  tissue  visualization  and  manipulation  can 
provide  results  that  are  at  least  as  good  as  those 
obtained  with  a  conventional  open  approach.  In 
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vascular  surgery,  percutaneous  endovascular  tech¬ 
niques  have  been  effective  for  treating  atherosclerosis 
with  methods  that  minimize  complications  and 
hasten  recovery. 

Although  a  number  of  techniques  and  instru¬ 
ments  currently  used  in  minimally  invasive  therapy 
were  proposed  years  ago,  the  ability  to  make  the 
transition  from  science  fiction  to  reality  required  the 
maturation  of  enabling  technologies.  Similarly,  ad¬ 
vances  in  computer,  communications,  robotics,  and 
display  technologies  have  enabled  the  development 
of  telepresence  surgery.^ 

The  fundamental  principle  of  a  telepresence 
surgery  system  is  to  extend  a  surgeon’s  psychomotor 
skills  and  problem-solving  abilities  to  a  remote 
environment.  The  goal  of  telepresence  technology  is 
to  project  a  surgeon’s  manual  dexterity  to  a  remote 
location  while  providing  real-time  tactile  and  visual 
feedback  from  the  location  to  the  surgeon.^  Unlike 
telerobotic  surgery,  in  which  a  robotic  manipulator  is 
directed  by  preprogrammed  computer  instructions, 
or  surgical  virtual  reality,®  in  which  manipulations  are 
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performed  in  a  simulated  environment,  telepresence 
is  a  unique  human-machine  technolog)^  that  directly 
and  transparently  projects  a  surgeon’s  motions  to  a 
remote  location.^ 

The  potential  applications  of  telepresence  surgery 
are  myriad.  It  would  allow  surgeons  to  perform 
surgery  in  patients  who  are  in  hazardous  environ¬ 
ments  or  who  have  been  victims  of  biologic  or 
chemical  warfare  agents.  Surgical  procedures  could 
be  safely  performed  during  prolonged  radiographic 
imaging.  It  could  provide  surgical  expertise  in 
situations  in  which  an  experienced  surgeon  is  not 
available,  such  as  enabling  life-saving  trauma  surger}^ 
in  rural  settings  or  on  a  battlefield.  Telepresence 
surgery  could  also  be  used  to  provide  expert  assis¬ 
tance  to  trained  surgeons  who  are  performing 
difficult  or  unusual  procedures  in  a  community 
hospital  and  are  unable  or  unwilling  to  transfer  a 
patient  to  a  tertiary  care  center. 

We  describe  the  first  studies  of  the  use  of 
telepresence  surgery  on  live  animals.  We  have  dem¬ 
onstrated  the  technical  feasibility  of  remote  tissue 
handling  by  performing  tissue  incision,  dissection, 
and  blood  vessel  suturing  completely  by  telepresence 
surgery. 

MATERIALS  AND  METHODS 

Telepresence  surgery  system.  The  telepresence 
surgery  system  used  in  these  studies  was  developed  by 
SRI  International.  It  is  a  unique  modular  system  that 
comprises  a  surgeon’s  workstation  and  a  remote 
surgical  unit  (RSU)  (Fig.  I).  The  workstation 
provides  a  fall-color  stereoscopic  (3D)  image  of  the 
operative  field  projected  in  real-time  fall-frame  video. 
Beneath  the  imaging  screen  are  two  surgical  instru¬ 
ment  handles  that  are  grasped  by  the  surgeon.  The 
ergonomic  design  is  such  that  a  completely  natural 
orientation  of  operative  field  and  hand  position  are 
attained.  Alsoincluded  are  stereophonic  audio  input, 
a  microphone  for  communicating  with  the  RSU,  and 
video  displays  of  the  remote  operating  room. 

The  RSU  is  positioned  over  the  operating  table  at 
a  location  removed  from  the  workstation.  The  only 
physical  connection  between  the  two  units  is  an 
electronic  data  cable,  which  could  be  replaced  by  a 
wireless  communications  link.  The  RSU  has  two 
high-resolution  digital  videocameras  with  electronic 
zoom  lenses  oriented  to  provide  3D  image  transmis¬ 
sion.  Beneath  the  cameras  are  servomotor-controlled 
manipulator  arms  with  interchangeable  surgical  in¬ 
strument  attachments  on  their  ends. 

Degrees  of  freedom  (DOF)  refers  to  the  number 
of  different  ways  that  a  robotic  arm  can  move.  The 


human  arm  has  7  DOF,  and  the  human  hand  has  over 
20  DOF.^  In  the  current  telepresence  surgery  system, 
the  manipulators  have  4  DOF,  plus  opening  and 
closing  of  the  actuators  (the  surgical  instruments 
attached  to  the  manipulators)  (Fig.  2).  Thus  the 
surgeon  operates  as  though  his  or  her  wrist  were 
immobilized.  The  movements  of  the  surgeon’s  hands 
at  the  workstation  are  precisely  and  instantly  trans¬ 
lated  to  movement  of  the  instruments  at  the  ends  of 
the  manipulator  arms  of  the  RSU.  Instruments 
currently  available  include  a  scalpel  handle,  needle 
holders,  curved  and  blunt  graspers,  and  fine  scissors. 

An  essential  component  of  the  telepresence  sur¬ 
gery  system  is  force  reflection,  the  built-in  tactile 
feedback  provided  to  the  surgeon  at  the  workstation.® 
Each  movement  made  by  the  surgeon  at  the  work¬ 
station  is  resisted  by  the  tissue  in  contact  with  the 
surgical  instruments  on  the  RSU,  The  resistance  is 
proportional  to  the  force  applied  and  the  character¬ 
istics  of  the  tissue  manipulated.  For  example,  when 
grasping  and  retracting  a  tissue,  the  sense  of  resis¬ 
tance  to  stretch  is  instantly  felt  by  the  surgeon  at  the 
workstation,  which  allows  him  or  her  to  reduce  the 
force  of  grasping  to  prevent  tearing.  This  sensation  is 
similar  to  that  achieved  during  tissue  manipulation  in 
conventional  open  surgery  and  is  essential  for  precise 
atraumatic  handling  of  tissues.  Because  it  reproduces 
the  normal  tactile  input,  surgery  with  the  telepres¬ 
ence  surgery  system  feels  natural  and  intuitive  to 
surgeons. 

Ex  vivo  studies.  Three -cm  incisions  were  made 
anteriorly  in  fresh  excised  segments  of  bovine  aorta. 
Conventional  techniques  were  used  to  close  the 
arteriotomies  with  continuous  running  suture  of  4-0 
polypropylene  or  Gore-tex  (W.L.  Gore  and  Asso¬ 
ciates;  Flagstaff,  Ariz.).  Tapered-point  needles  were 
used. 

Telepresence  closure  of  arteriotomies  were  per¬ 
formed  by  a  surgeon  at  the  workstation.  Assistance 
was  limited  to  suture-cutting  by  a  nonphysician 
assistant  located  at  the  RSU.  A  suture  was  placed  in 
the  operative  field  by  the  surgical  assistant,  after 
which  all  tissue  manipulation  was  performed  by  the 
surgeon  at  the  workstation.  All  procedures  were 
recorded  in  real  time  on  videotape.  One  of  the  stereo¬ 
scopic  cameras  on  the  RSU  was  used  for  recording 
telepresence  surgical  procedures,  and  a  scene  camera 
was  used  for  conventional  procedures. 

Time-and-motion  analysis  was  conducted  by  a 
complete  review  of  all  videotapes  by  the  senior 
clinical  author  (JCB).  Times  for  suture  placement 
(start  of  needle  entry  into  tissue  until  removal  from 
tissue  at  the  end  of  the  bite),  positioning  (needle 
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Fig.  1.  Telepresence  surgery  system.  A,  Surgeon’s  workstation  with  3D  viewing  screen  and 
instrument  handles.  B,  RSU  with  manipulator  arms. 


removal  from  tissue  to  start  of  next  bite),  and  knot 
tying  were  recorded. 

In  vivo  studies.  All  animal  care  complied  with 
the  ‘Trinciples  of  Laboratory  Animal  Care”  (formu¬ 
lated  by  the  National  Society  for  Medical  Research), 
and  the  Guide  for  the  Care  md  Use  of  Laboratory 
Animals  (NIH  Publication  No.  86-23,  revised  1985). 
General  anesthetic  was  used  in  female  swine  weigh¬ 
ing  35  to  50  kg.  Noninvasive  cardiac,  respiratory,  and 
blood  pressure  monitoring  was  performed  through¬ 
out  the  course  of  the  experiments. 

Common  femoral  arteries  were  exposed  and 
isolated  by  the  telepresence  surgeon,  and  100  U/kg 
heparin  was  administered  intravenously.  After  the 
surgeon  occluded  the  femoral  artery  by  applying 
traction  to  vessel  loops,  a  3-cm  arteriotomy  was 
made.  6-0  polypropylene  or  polytetrafluoroethylene 
(PTFE)  suture  on  a  tapered  needle  was  used  to  close 
the  arteriotomy  in  a  continuous  running  suture.  As  in 
ex  vivo  experiments,  assistance  was  limited  to  cutting 
suture.  At  the  completion  of  the  arteriotomy  closure, 
clamps  were  removed.  Hemostasis  was  achieved  with 
topical  Gelfoam  (Upjohn,  Kalamazoo,  Mich.)  or 
fibrin  adhesive  (American  Red  Cross,  Rockville, 
Md.)  held  in  place  for  2  minutes  by  the  surgeon 
working  through  the  telepresence  surgery  system. 
Patency  was  confirmed  by  continuous-wave  Doppler 


insonation  conducted  immediately  after  and  1  hour 
after  repair  and  by  transecting  the  femoral  artery 
distal  to  the  repair  at  the  completion  of  the  study  and 
observing  pulsatile  blood  flow.  Repaired  arterial 
segments  were  then  excised  and  opened.  The  pres¬ 
ence  of  thrombus  and  the  integrity  of  the  suture  line 
were  noted. 

RESULTS 

Ex  vivo  experiments.  Conventional  closure  of  a 
3-cm  arteriotomy,  including  knot  tying  at  the  begin¬ 
ning  and  end  of  the  suture  line,  required  192  ±  24 
sec  (n  =  4) .  Arteriotomy  closure  performed  with  the 
telepresence  surgery  system  required  483  ±  118  sec 
(n  =  9).  All  procedures  by  either  technique  were 
technically  acceptable  as  assessed  by  integrity  to 
pressure  perfusion,  patency,  and  intraluminal  evalu¬ 
ation  of  the  suture  line. 

With  conventional  techniques,  79%  of  the  time 
was  spent  placing  sutures  and  21%  tying  knots.  With 
telepresence  surgery,  76%  of  time  was  spent  placing 
sutures  and  24%  tying  knots.  Telepresence  surgery 
required  more  time  for  grasping  and  repositioning 
the  needle  between  each  suture  than  did  conventional 
suturing. 

Eliminating  force  feedback  (tactile  sensation)  and 
3D  imaging  resulted  in  a  16%  to  20%  mean  increase 
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Fig.  2.  DOF  describe  the  planes  in  which  manipulator 
arms  can  be  moved.  4  DOF  plus  actuator  (instrument  tip) 
opening  and  closing  are  available  in  the  current  telepresence 
surgery  system. 

in  the  time  required  to  complete  the  procedure  with 
telepresence  surgery,  although  the  technical  quality 
was  comparable  in  each  case.  Subjectively,  lack  of 
force  feedback  and  3D  imaging  eliminated  the 
intuitive  sensation  and  increased  operator  fatigue. 
This  was  most  noticeable  when  working  on  small- 
diameter  vessels,  in  which  the  ability  to  complete 
procedures  was  severely  compromised. 

Other  ex  vivo  procedures  performed  with  bovine 
aortae  were  patch  angioplasty  with  a  0.4-mm  PTFE 
cardiovascular  patch  (n  =  4)  and  end-to-side  PTFE 
patch  to  aorta  anastomosis  performed  with  6- mm 
thin-walled  PTFE  (n  =  4).  The  procedures  were 
completed  with  telepresence  surgery  with  technically 
acceptable  results.  That  the  RSU  manipulator  had 
only  4  DOF  required  rotation  of  the  aorta  to 
complete  the  procedures  —  a  limitation  that  would  be 
overcome  with  additional  manipulator  DOF. 


In  vivo  femoral  artery  repair.  With  a  continu¬ 
ous  suture,  2-cm  lacerations  in  common  femoral 
arteries  were  closed  successfully  by  telepresence 
surgery  with  the  starting  suture  placed  in  a  mattressed 
fashion.  Arteriotomy  closure  required  1388  ±  122 
sec  (n  =  6)  with  the  telepresence  surgery  system.  All 
closures  performed  by  telepresence  surgery  were 
technically  adequate,  patent,  and  without  suture-line 
bleeding.  In  no  instance  did  excessive  tension  result 
in  a  tear  in  the  blood  vessel  or  a  break  in  the  suture. 

DISCUSSION 

More  than  4  decades  ago,  harnessing  atomic 
energy  required  methods  for  safely  handling  danger¬ 
ous  amounts  of  radioactive  materials.  Separating 
nuclear  workers  from  contaminated  environments 
was  first  accomplished  by  creating  mechanical  links  to 
manipulate  tools  used  in  hazardous  spaces.^  This 
method  was  effective  for  very  specific  tasks  over  short 
distances  and  is  occasionally  used  today.  True  remote 
manipulation,  or  teleoperation,  was  developed  to 
handle  large  items  that  were  physically  remote  from 
the  operator.  These  manipulator  systems  were  cum¬ 
bersome  and  relied  only  on  visual  input  to  the 
operator.  Although  they  were  effective  for  handling 
large  objects  in  tasks  that  required  only  moderate 
accuracy,  the  lack  of  tactile  feedback  hindered  the 
ability  to  precisely  position  small  objects  or  perform 
intricate  tasks. 

Telepresence  manipulators  provided  tactile  sen¬ 
sory  input  to  the  operator  by  providing  force 
feedback,  which  is  a  sense  of  resistance  encountered 
by  the  manipulator  arms  when  contacting  objects  in 
the  remote  environment.  Force  feedback  resulted  in 
more  rapid  and  accurate  task  completion  and  de¬ 
creased  operator  fatigue.  With  force  feedback,  tasks 
that  required  greater  precision  could  be  per- 
formed.^^’^^ 

In  the  1970s,  the  National  Aeronautics  and  Space 
Administration  began  developing  telepresence  sys¬ 
tems  for  surgery  in  space.  The  anticipated  construc¬ 
tion  and  operation  of  a  manned  space  station 
required  contingency  planning  for  emergency  surgi¬ 
cal  care  in  a  location  where  specialists  would  be 
unavailable.  Although  the  spacestation  was  never 
built,  the  potential  of  remote  telepresence  surgery  in 
a  broad  spectrum  of  applications  was  recognized. 
Continued  research  and  the  maturation  of  necessary 
supporting  technologies  facilitated  the  development 
of  a  complete  integrated  system  that  could  be  used  in 
preclinical  studies.  Our  goal  was  to  demonstrate  the 
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feasibility  of  using  a  telepresence  surgery  system  to 
perform  common  procedures  that  require  the  cog¬ 
nitive  and  motor  skills  of  a  trained  surgeon.  The 
dissection,  handling,  and  suturing  of  tissue  are  tasks 
common  to  all  surgical  disciplines.  Vascular  anasto¬ 
moses  and  the  closure  of  small-vessel  arteriotomies 
require  psychomotor  integration  of  basic  skills  and 
represent  reasonable  clinical  criteria  for  judging  the 
system’s  effectiveness.  On  the  basis  of  these  criteria, 
telepresence  surgery  was  successful.  In  remote  or 
battlefield  applications  of  telepresence  surgery,  a 
nonphysician  assistant  would  be  trained  to  assist  the 
telepresence  surgeon.  For  this  reason,  assistance 
during  our  studies  was  limited  to  technical  skills  that 
a  surgical  technician  is  currently  trained  to  provide. 
These  skills  include  tissue  grasping,  suture  cutting, 
and  retraction. 

A  primary  objective  in  designing  complex  systems 
that  require  operator  control  or  input  is  creating  an 
ideal  human  interface.  The  ideal  interface  is  one  that 
feels  completely  natural,  or  intuitive,  to  the  user.^^ 
Laparoscopic  surgery  is  not  natural  because  the 
eye— hand  axis  normally  used  in  surgery  and  everyday 
tasks  is  disrupted.  Instead  of  looking  down  at  his  or 
her  hands  while  working,  the  laparoscopic  surgeon  is 
forced  to  look  up  at  a  screen,  disrupting  normal 
oculovestibular  sensory  input. A  further  limitation 
of  laparoscopic  surgery  is  the  use  of  long  instruments 
with  the  fulcrum  located  at  the  abdominal  wall,  far 
from  the  tissues  being  manipulated.  Tactile  feedback 
is  lost,  and  precise  tissue  manipulation  is  more 
difficult.  Consequently,  laparoscopic  surgical  skills 
need  to  be  learned  and  reinforced  by  regular  practice. 

We  found  the  telepresence  surgery  system  to  be 
remarkably  similar  to  conventional  surgery.  No 
special  training  was  conducted  before  using  the 
system,  and  the  time  required  for  suturing  and  tying 
did  not  decrease  substantially  with  use.  Subjective 
comments  about  how  closely  telepresence  techniques 
resembled  conventional  techniques  add  further  sup¬ 
port  to  this  contention.  Almost  invariably  a  surgeon 
using  the  telepresence  system  would  at  some  point 
remove  a  hand  from  the  instrument  handle  and 
attempt  to  retract  tissue  as  if  the  operative  field  was 
right  ffiere.  The  successful  closure  of  arteriotomies  in 
2-  to  3-mm  diameter  vessels  demonstrated  the 
feasibility  of  telepresence  surgery  in  a  procedure  that 
requires  a  high  degree  of  integration  of  technical 
skills.  We  believe  that  the  longer  times  required  for 
task  completion  by  telepresence  surgery  reflect  the 
reduced  DOF  available  in  the  current  system  and  the 


limitations  imposed  by  the  current  selection  of 
instruments.  The  needle  holders  and  forceps  used 
were  larger  than  optimal  for  fine  vascular  suturing, 
which  made  needle  grasping  and  knot  tying  more 
difficult.  Vascular  instruments  are  currently  being 
adapted  for  use  with  the  telepresence  surgery  system. 

Stereoscopic  vision  and  force  reflection  contrib¬ 
uted  to  the  intuitive  interface.  The  observation  that 
simple  suturing  could  be  performed  without  either 
feature  with  only  a  modest  increase  in  task- 
completion  times  most  likely  reflects  the  ability  of 
the  human  visual  system  to  use  alternative  sources 
of  depth  information  when  stereoscopic  vision  is 
lost.  Other  cues,  including  perspective,  interposi¬ 
tion,  contrast,  object  size,  and  shadows  become  of 
primary  importance.®  Of  equal  significance  is  that 
procedures  became  tedious  and  fatigue  developed 
quickly.  Suturing  of  small  vessels  was  so  difficult  as 
to  be  impractical.  Further  studies  of  visual  and 
tactile  cues  are  required  to  define  the  optimal  user 
interface. 

The  current  telepresence  surgery  system  has  only 
four  DOF  in  the  manipulator  arms.  Thus  the 
potential  range  of  instrument  movements  simulates  a 
human  arm  with  a  fused  (nonmobile)  wrist.  Al¬ 
though  the  system  is  adequate  for  demonstrating 
feasibility,  wrist  function  will  need  to  be  added  for  it 
to  achieve  widespread  acceptance  in  clinical  use.  The 
next-generation  system  will  provide  a  foil  7  DOF  in 
the  manipulator  arms.  With  this  addition,  a  surgeon 
will  be  capable  of  performing  virtually  all  hand 
movements  used  in  the  operating  room. 

The  obvious  uses  of  telepresence  surgery  are  in 
the  extension  of  the  expertise  of  a  surgeon  to 
locations  not  currently  possible.  In  the  military, 
hemorrhage  is  the  primary  cause  of  death  on  the 
battlefield.  Most  casualties  exsanguinate  within  30 
minutes  of  wounding.  Although  it  is  impossible 

to  have  enough  surgeons  on  the  front  lines  in  battle, 
it  is  feasible  to  have  an  RSU  accompanying  the 
medics  who  first  encounter  wounded  soldiers.  These 
medics  will  have  the  training  to  provide  surgical 
assistance  and  perform  paramedic  skills,  including 
intubation  and  initiation  of  intravenous  access.  Par¬ 
allel  development  programs  are  exploring  methods 
for  providing  anesthetic  support  to  distant  locations 
with  remote  monitoring,  artificial  intelligence,  and 
new  drugs.  Skilled  surgeons  operating  from  estab¬ 
lished  field  hospitals  or  even  from  the  United  States 
could  rapidly  perform  life-saving  stabilizing  surgery. 
Telepresence  surgery  could  help  provide  the  first 
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reduction  in  early  battlefield  mortality  in  more  than 
150  years. 

Another  unique  application  of  telepresence  sys¬ 
tems  is  the  performance  of  emergency  surgery  in  an 
hazardous  environment  or  on  a  casualty  contami¬ 
nated  by  biologic  or  chemical  warfare  agents.  With 
such  systems,  the  current  limitations  imposed  by 
chemical  protective  suits  or  the  delays  imposed  by 
patient  decontamination  could  be  overcome.  Re¬ 
moving  unexploded  munitions  from  body  cavities  is 
another  potential  use  with  both  military  and  civilian 
applications.^^ 

Advanced  telepresence  surgery  systems  could  be 
used  in  minimally  invasive  surgery  and  would  return 
the  instrument  fulcrum  and  dexterity  to  the  tissue 
level.  This  would  facilitate  vascular  anastomoses 
through  small  trocar  incisions  and  permit  some 

procedures  iiow  done  by  conventional  lc.  hniqu^ . 

be  performed  lessinvasively.^^ 

Because  all  hand  and  manipulator  movements 
are  digitally  processed,  force-reduction  algorithms 
can  be  introduced  that  would  allow  movements  of 
a  surgeon’s  hands  to  be  scaled  down  at  the  tissue 
level.  Thus  a  10-mm  hand  excursion  by  the  surgeon 
could  be  translated  to  0.1  mm  or  even  less  at  the 
tissue  level.  Microvascular  procedures  could  be  per¬ 
formed  on  a  level  that  is  not  possible  with  con¬ 
ventional  techniques.  The  same  approach  could  be 
used  to  dampen  incorrect  movements  or  tremors, 
which  would  allow  novice,  aging,  nervous,  fatigued, 
or  less-dexterous  surgeons  to  perform  vascular  pro¬ 
cedures.^^ 

Telepresence  surgery  systems  will  require  further 
evolution  and  testing  before  they  extend  a  surgeon’s 
reach  to  a  remote  operating  room.  Most  important  in 
the  development  process  is  a  sound  clinical  founda¬ 
tion.  By  demonstrating  that  fundamental  procedures 
in  vascular  surgery  can  be  performed  by  a  remotely 
located  surgeon,  we  have  taken  the  first  steps  to 
acquiring  a  revolutionary  new  tool  for  the  21st 
century  surgeon. 

The  assistance  of  Lawrence  W.  Way,  MD,  and  the 
Experimental  Surgery  Laboratory  of  the  University  of 
California,  San  Francisco  is  gratefully  acknowledged. 

Technical  details  of  the  SRI  Telepresence  Surgery^ 
System  and  videotapes  of  procedures  performed  are 
available  from  the  Medical  Technology  Laboratory,  SRI 
International,  333  Ravenswood  Drive,  Menlo  Park,  CA 
94025. 
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DISCUSSION 

Dr.  Sushil  K.  Gupta  (Framingham,  Mass.).  I  con¬ 
gratulate  Dr.  Bowersox  for  reaching  an  important  mile¬ 
stone  in  a  journey  that  began  two  decades  ago  ~  a  journey 
to  perform  surgery  in  space  by  remote  control. 

The  goals  of  this  project  are  noble— to  operate  on 
patients  in  distant  locations  or  hazardous  environments.  I 
do  not  want  to  belittle  the  authors’  significant  achievement 
to  date  by  asking  them  simple  questions  such  as  who  will 
prep  and  drape  the  patient,  who  will  administer  anesthetic, 
and  who  will  then  care  for  the  patient.  It  is  a  significant 
achievement  to  be  able  to  tie  a  6-0  polypropylene  suture 
with  a  remotely  controlled  robotic  device.  I  can’t  even  get 
our  residents  to  do  this  without  breaking  or  entangling  the 
suture  while  I’m  standing  at  the  table. 

I  empathize  with  fellow  members  in  the  audience  who 
are  wondering:  With  the  radiologists  encroaching  on  our 
business  on  one  end  and  now  robots  appearing  at  the  other 
end,  isn’t  it  time  to  retire? 

Dr.  Bowersox  will  acknowledge,  however,  that  signif¬ 
icant  challenges  are  still  ahead.  I  have  some  performance 
concerns.  Is  there  any  lag  time  now  with  the  current  system 
tethered  at  600  feet  and  only  4  DOF  in  the  robotic  arm? 
Whzt  will  happen  to  the  bandwidth  and  throughput  when 
you  go  wireless  and  add  7  DOF  and  better  optics?  Are  there 
plans  to  add  head-mounted  motion  systems  to  simulate 
head  movement  to  look  in  deeper  or  fix  the  posterior  wall 
injury  that  is  all  too  common?  Would  it  not  increase  the 
digital  communication  requirements  such  that  it  would 
prevent  real-time  surgery?  In  a  recent  experiment,  the 
network  control  of  teleoperators  not  too  far  away  had 
significant  lag  times  of  50  msec  to  several  seconds,  which 
makes  some  tasks  impossible  to  carry  out. 

I  also  disagree  with  the  authors’  contention  that  this 
approach  could  be  used  to  dampen  incorrect  movement  or 
tremors,  allowing  novice,  aging,  nervous,  or  less-dexterous 
surgeons  to  perform  vascular  procedures.  I  do  agree  that 
further  development  of  this  kind  will  greatly  enhance  the 
field  of  minimally  invasive  surgery  and  allow  vascular 
surgeons  to  perform  some  procedures  with  laparoscopic 
techniques  by  improvements  in  the  vision  and  the  instru¬ 
mentation  to  carry  out  more  delicate  tasks. 

Dr.  Bowersox.  Thank  you  for  your  insightful  review 
and  pertinent  questions.  I’d  like  to  emphasize  from  the  start 
that  the  telepresence  surgery  program  is  one  small  com¬ 
ponent  of  a  larger  program  sponsored  by  the  Advanced 


Research  Projects  Agency  and  supported  by  the  National 
Institutes  of  Health  in  Advanced  Technology  in  all  areas  of 
biomedicine,  and  I  think  that  information  will  provide  the 
answers  with  regard  to  how  this  system  is  going  to 
integrate  with  the  access  to  anesthetic  and  to  physiologic 
monitoring. 

Once  again,  I’m  a  clinician  and  not  a  computer  scientist 
or  engineer,  but  I’d  like  to  briefly  review  some  of  the 
questions  you  raised  about  the  technical  issues  of  the 
system.  Lag  time  is  certainly  an  issue  when  you  deal  with 
satellite  communications.  Rudimentary  studies  have 
shown  that  if  you  introduce  more  than  100  msec  of  lag 
time,  it  is  very  difficult  to  do  surgery  with  accuracy  and 
precision.  This  difficulty  does  not  occur  if  you  remain  in  a 
fiberoptic  environment  or  even  over  a  microwave  link.  So 
the  quick  answer  to  that  question  is  it  can  be  done  with 
current  technology  at  distances  up  to  but  not  including 
satellite  links.  We  can  go  across  country  or  even  across  a 
continent  with  fiberoptic  communications.  A  solution  is 
underway  with  software  development  that  will  allow  this 
connection  to  go  over  a  satellite  link,  although  probably  not 
for  the  next  several  years. 

The  4  DOF  is  a  limitation;  it’s  currently  as  though  you 
were  operating  with  a  fused  wrist.  A  prototype  of  a 
manipulator  with  7  DOF  has  been  built  and  should  be 
ready  for  testing  by  the  end  of  1995.  This  prototype  will 
allow  all  motions  of  the  human  arm  to  be  modeled  and 
should  remove  the  limitation  imposed  by  having  only  four 
DOF. 

The  bandwidth  requirements  are  modest.  From  a 
technical  perspective,  45  megabits/sec  should  provide  the 
bandwidth  necessary  for  this  system.  Interestingly,  most  of 
that  requirement  is  for  the  stereo  video,  which  can  be 
accommodated  by  current  fiberoptic  networks  with  no 
compression.  That  the  bandwidth  is  not  a  limitation 
surprised  me  when  I  first  started  working  on  the  system. 
The  actual  manipulator  and  tactile  feedback  occupies  a  very 
small  portion  of  that  bandwidth,  only  1  megabit/sec,  which 
won’t  increase  substantially  when  going  to  4  DOF. 

The  head-mounted  display  is  a  very  intriguing  concept; 
we’re  testing  it  now  in  the  laboratory.  Will  this  have  a  role 
for  the  aging,  disabled,  or  tremulous  surgeon?  I’m  not  sure, 
but  I  do  know  that  this  system  could  be  used  to  overcome 
those  limitations,  and  I  think  it’s  something  that  we  as 
clinicians  need  to  explore  and  make  a  decision  about. 
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ABSTRACT 

Endoscopic  surgical  methods  are  replacing  open 
surgery  in  many  procedures,  but  dexterity  and  force 
feedback  are  not  adequate  with  current  tools.  To  enhance 
the  ability  of  surgeons  to  operate  endoscopically,  we  have 
developed  a  telepresence  system  with  integrated  3D  stereo 
viewing,  a  prototype  force-reflecting  manipulator,  and 
aural  feedback.  Careful  attention  was  paid  to  the  human 
factors  in  the  endoscopic  surgery  setting  to  make  the  system 
natural  to  use  in  the  hope  of  eliminating  the  long  training 
period  normally  required.  The  4-axis  (plus  gripper) 
manipulator  provides  the  same  degrees  of  freedom  as  the 
lap  aroscopic  tools  now  being  used  for  surgery.  The 
bilateral  control  system  provides  for  magnified  motion  and! 
or  force  r^ection.  This  approach  eliminates  the  motion 
reversal,  or  fulcrum  effect,  in  operating  through  the 
abdominal  wall.  Preliminary  dexterity  experiments  with 
different  force  feedback  and  viewing  conditions  verify 
intuitive  use  and  fast  learning.  Recent  developments  with 
dexterous,  two-handed  operation  are  reported. 

L  INTRODUCTION 

During  the  past  several  years,  laparoscopic  surgery  has 
largely  replaced  conventional  open  surgery  for  removal  of 
the  gallbladder  and  has  begun  to  make  inroads  into  other 
procedures  as  well.  However,  the  dexterity  and  touch 
sensitivity  that  laparoscopic  instruments  provide  are  quite 
poor  compared  to  those  of  open  surgery,  because  the 
instruments  are  operated  through  an  incision  in  the 
abdominal  wall  which  serves  as  a  fulcrum,  with  the 
surgeon’s  hand  on  the  opposite  side  of  the  fulcrum,  away 
from  the  surgical  site.  Moreover,  the  video  image  engenders 
a  feeling  of  disconnectedness.  Consequently,  every  motion 
is  deliberate  rather  than  intuitive,  and  relatively  simple 
maneuvers,  such  as  suturing,  are  time  consuming  and 
frustrating. 


In  an  effort  to  enhance  the  ability  of  surgeons  to  operate 
laparoscopically,  we  have  developed  a  telepresence  surgery 
system  with  integrated  stereo  viewing,  force-reflecting 
manipulators,  and  aural  feedback. This  remote  surgery 
system  provides  the  surgeon  with  the  sense  that  the  surgical 
instruments  are  actually  in  his  or  her  hand.  When  applied  to 
laparoscopic  surgery,  it  provides  the  sense  of  doing  open 
surgery.  To  accomplish  these  improvements,  careful 
attention  was  paid  to  the  human  factors  involved  in  surgery 
and  to  the  special  demands  of  laparoscopic  surgery. 

The  new  telepresence  demonstration  system 
incorporates  two  manipulators,  each  with  the  same  four 
degrees  of  freedom  (plus  tip  actuation)  used  in  laparoscopic 
instruments  now  in  use.  However,  with  telepresence,  the 
surgeon’s  hand  is  effectively  on  the  correct  side  of  the 
fulcrum.  The  bilateral  control  system  provides  for 
magnified  motion  and/or  force  reflection.  The  low-inertia 
electro-mechanical  design  provides  a  high-bandwidth, 
low-drag  system  with  surprising  realism.  System  operation 
is  intuitive  and  natural;  no  operator  training  is  required  and 
remote  tasks  can  be  performed  with  a  high  degree  of 
dexterity.  Preliminary  dexterity  experiments  have  been 
conducted  with  various  force  feedback  and  viewing 
conditions. 

n.  BACKGROUND 

Robots  have  begun  to  find  applications  in  surgery  in 
recent  years.^  For  example,  placement  of  probes  in  the 
brain,  eye,  and  spinal  cord  has  been  demonstrated  using 
industrial  robots  such  as  the  Unimation  Puma  260. 
Typically,  the  robot  inserts  a  needle  along  its  axis  deep  into 
a  tumor.  The  Robodoc^  orthopaedic  surgery  system, 
developed  at  IBM  and  Integrated  Surgical  Systems. 


^  All  product  names  mentioned  in  this  document  are  the 
trademarks  of  their  respective  holders. 


produces  a  precise  cavity  in  the  femur  for  hip  replacement. 
TransurethM  resection  of  the  prostate  (TURP)  has  been 
performed  robotically.^  ^ 

In  contrast  to  robot  automation,  others  have  envisioned 
the  potential  advantages  of  telemanipulation  in  surgery. 
Alexander^^  shows  a  concept  for  a  teleoperator  surgery 
system  in  which  the  operator  could  see  and  feels  what  he  is 
operating  on.  Thring^^  describes  a  bilateral  surgical 
telerobot  concept  in  which  exoskeletal  masters  would  be 
used  to  operate  a  pair  of  xyz  Cartesian  manipulators. 

Marsuhima  and  Koyanagi^^  describe  a  hand-held 
master  with  a  tool-like  portion,  and  Sabatini,  Bergamasco, 
and  Dario^^  describe  cutting  with  a  finger-sized 
manipulator  (4  links)  proposed  for  comeal  transplant 
surgery.  Gayed  et  al.^^  and  Guerrouad  and  Vidal^^  have 
described  the  use  of  a  sterotaxic  manipulator  with  a  remote 
center  of  rotation,  for  placing  a  probe  into  the  eye.  They 
used  a  six-axis  manipulator  with  joystick  controls  that 
lacked  force  feedback.  Charles  et  al.^^  describe  the 
human-interface  requirements  for  microsurgical  telerobot 
surgery. 

With  the  increasing  use  of  laparoscopic  surgical 
techniques,  there  is  now  a  strong  need  for  telemanipulators 
that  can  perform  the  full  spectmm  of  surgical  maneuvers, 
e.g.,  cutting,  suturing,  dissecting,  normally  performed  by 
surgeons.  Instruments  with  the  required  dexterity,  speed, 
and  delicate  force  feedback  have  not  been  previously 
developed,  nor  has  a  methodology  for  making  their  use 
natural  and  effective. 

in.  TECHNICAL  APPROACH 
A.  SRI  Telepresence  System 

Our  current  demonstration  system,  diagrammed  in 
Figxu-e  1,  consists  of  two  modules:  the  Operator  Module  and 
the  Worksite  Module.  The  operator  module,  as  shown  in 
Figure  2,  provides  a  pair  of  force-reflecting  hand  controllers 
and  a  stereo  display  system  with  adjustable  magnification. 
Typically,  we  use  a  field  of  view  of  about  10  cm  by  14  cm, 
twice  the  size  of  the  video  field  in  the  worksite  module; 
however,  much  greater  magnification  is  possible.  The 
operator  module  contains  a  color  monitor,  a  Tektronbc 
liquid-crystal  shutter  for  stereogr^hic  viewing  (the 
operator  wears  passive  polarized  glasses),  a  mirror  to  create 
a  “virtual  workspace”  beneath  the  hand-operated  master 
controller,  stereo  speakers,  and  the  master,  which  is 
outfitted  with  a  surgical  instrument  handle. 


Figure  1.  SRI  Telepresence  Surgery  System 


Figure  2.  Operator  Module 

The  worksite  module,  shown  in  Figure  3,  provides 
stereopsis  with  two  color  CCD  video  cameras  separated  by 
about  10°,  the  same  interocular  viewing  disparity  that  we 
experience  with  a  visual  field  40  cm  in  front  of  us. 

Also  included  are  a  force  reflecting  manipulator  with 
surgical  instrument  tips  and  two  microphones. 


C.  Force-Reflecting  Manipulators 


Figure  3*  Worksite  Module 
B.  Telepresence  Human  Factors 

The  master  and  slave  manipulators  are  placed  in 
juxtaposition  to  the  camera  and  to  the  display  respectively 
so  as  to  be  geometrically  identical.  Our  purpose  is  to 
effectively  transport  the  remote  site  back  to  the  operator  in 
such  a  way  as  to  create  the  illusion  that  he  or  she  is  reaching 
and  looking  directly  into  it.  This  results  in  natural  and 
spontaneous  control  motions  by  the  operator. 

Telepresence  is  a  unique  method  of  remote 
manipulation.  The  telepresence  surgeon  grasps  surgical 
instrument  handles,  looks  down  into  a  three-dimensional 
surgical  field,  reaches  into  the  field  with  the  instruments, 
and  operates.  He  sees  the  tips  of  the  instruments  move  with 
his  hands  as  they  manipulate  the  tissues,  and  feels  the  tissues 
resist.  To  the  surgeon,  this  looks  and  feels  like  the 
conventional  open  surgery  he/she  has  already  trained  and 
practiced  on.  The  patient  could  be  in  the  same  room  or 
100  kilometers  (60  miles)  away:  it  makes  no  difference. 
Telepresence  provides  such  a  compelling  sense  of  reality 
that  the  surgeon  is  drawn  immediately  into  the  work,  with 
no  sensation  of  remote  control.  Hand  motions  are  quick  and 
precise.  Furthermore,  the  visual  field,  instrument  motion, 
and  force  feedback  can  be  scaled  up  or  down;  for  example, 
to  make  microsurgery  even  easier  than  it  would  be  if  the 
surgeon  were  at  the  patient’s  side,  operating  directly. 


One  master,  or  system  controller,  is  shown  in  Figure  4. 
The  handgrips  consist  of  the  loop  handles  of  a  pair  of 
surgical  forceps.  An  extension  joint  supports  the  handles, 
which  in  turn  is  supported  by  a  three-axis  gimbal,  providing 
free  rotation  on  its  three  angles.  The  loop  handles  extend  and 
retract  along  its  linear  extension  axis  and  open  and  close. 
Small  electric  motors  attached  to  each  axis  by  miniature 
timing  belts  reflect  magnified  forces  from  the  slave 
manipulator  back  to  the  instrument  handle  in  the  operator’s 
hand,  allowing  multiaxial  forces  on  the  manipulator  to  be 
perceived  in  a  completely  natural  way.  Each  slave  (surgical 
manipulator)  is  a  similar  articulated  device,  shown  in 
Figure  5. 

By  keeping  the  mechanism  very  light  and  the  motors 
centrally  located,  we  have  minimized  the  inertia  of  the 
system  felt  by  the  operator.  We  have  also  minimized  friction 
by  using  few  stages  of  gearing,  low  friction/low  inertia 
actuators,  and  low  friction  bearings  throughout.  The  usual 
drawbacks  of  inertia  and  friction — ^reduced  operator 
performance  and  fatigue  after  prolonged  usage — have  been 
largely  eliminated. 

Force  feedback  is  provided  on  all  axes  of  the 
manipulator,  including  the  gripper  squeeze  force.  Forces 
and  torques  generated  as  the  tool  touches  objects  in  the 
course  of  its  work  are  reproduced  with  great  fidelity  at  the 
operator’s  hand.  Soft  objects  feel  soft;  hard  ones  hard.  The 
constraints  of  sliding,  inserting,  and  bottoming  out  are  clear 


Figure  4*  Master  Hand  Control 


Figure  5.  Slave  Manipulator 


Figure  6.  Medical  End  Effectors 


and  natural.  This  will  be  important  in  surgery;  currently 
used  laparoscopic  surgical  instruments  lack  adequate  force 
sensitivity. 

The  following  performance  of  the  manipulator  has  been 
obtained. 


Interchangeability  is  obtained  by  a  collet  near  the 
middle  portion  of  the  end-effector  connecting  rod.  A  clamp 
nut  is  unscrewed  a  few  turns,  and  the  collet  frees  the  outer 
connecting  rod.  A  ball  and  socket,  easily  separated,  link  the 
inner  push  rod.  Tools  can  be  changed  in  a  few  seconds. 


•  Frequency  response  (natural  frequency)  exceeds  20  Hz 
on  all  axes  (the  human  limit  is  about  10  Hz.) 

•  Force  levels  of  7  N  (approximately  1.5  pounds)  can  be 
applied  continuously  in  any  direction. 

•  Friction  is  limited  to  0.3  N  (approximately  1  ounce)  with 
the  master-slave  loop  closed. 

D.  Interchangeable  End-Effectors 

A  family  of  interchangeable  surgical  end  effectors  are 
provided,  including  graspers,  needle  drivers,  scalpels,  and 
scissors,  as  shown  in  Figure  6.  The  tool  tips  are  adapted 
from  actual  surgical  instruments  and  are  driven  via  a  push 
rod  in  the  center  of  the  connecting  rod  attaching  the  tool  tips 
to  the  manipulator.  To  obtain  high  grasping  forces  on  the 
needle  holder  end  effector,  we  employ  a  tool  with  short, 
carbide  inset  jaws.  Gripping  forces  of  between  50  to  100  N 
(10  to  20  lb)  are  obtained  for  good  grasp  on  the  needle.  The 
scalpel,  which  may  be  outfitted  with  standard  surgical 
blades,  is  not  actuated  by  the  grip  mechanism. 


E.  Manipulator  Control  System 

The  control  system  for  the  manipulator  is  shown  in 
Figure  7.  This  is  basically  a  position  and  velocity  (PV) 
control  system  similar  to  that  employed  by  Goertz^^  yj 
early  work,  with  bilateral  manipulators  for  handling 
radioactive  materials. 

The  control  system  is  entirely  digital;  it  faithfully 
reproduces  hand  motion  and  reflects  forces  to  the  operators 
hand.  Motion  and  force  feedback  can  be  scaled  to  suit  the 
task  at  hand.  Dynamic  gravity  compensation  is  provided, 
making  the  tool  feel  weightless. 
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Figure  7.  Manipulator  Control  Diagram 


F.  Stereographic  Visualization 

The  imaging  subsystem  is  shown  in  Figure  8.  Two 
standard  NTSC  video  cameras  provide  the  stereoscopic 
image  pair.  A  Datacube  dual,  digital  frame  buffer  alternately 
provides  each  image  to  the  operator’s  display,  a  Tektronix 
frame-sequential  stereo  system  with  a  120  frame/s  TV 
monitor  and  a  synchronously  switched  polarizing  filter.  The 
operator  wears  passive,  polarized  glasses,  which  pass 
alternate  frames  to  the  left  and  right  eye  respectively.  The 
doubled  frame-rate  ensures  flicker-free  stereo. 

G.  Aural  Feedback  System 

The  current  worksite  module  incorporates  two 
microphones  positioned  above  and  to  the  sides  of  the 
worksite  itself.  Two  speakers  are  located  in  the  operator 
module  just  above  and  to  the  sides  of  the  hand-controller 
region — the  apparent  worksite.  This  arrangement  provides 
stereophonic  sounds  that  seem  to  emanate  from  the  worksite 
and  that  are  generally  associated  with  the  side  of  the 
worksite  from  which  they  originated.  The  distance  between 
the  left  and  right  microphone  elements  is  approximately  the 
same  as  the  distance  between  the  speakers  that  reproduce 
the  sound. 

in.  PERFORMANCE  EVALUATION: 

ONE-HANDED  OPERATION 

We  have  conducted  informal  performance  tests  to 
assess  the  capabilities  of  the  telepresence  system  using  only 
one  hand.  It  is  remarkably  easy  to  perform  tasks  without 
practice  by  merely  grasping  the  instrument  handle  (the 
master)  and  doing  them.  With  the  assistance  of  an  assistant 
holding  or  tensioning  tissue,  we  have  demonstrated  surgical 
dissection  of  both  excised  animal  muscle  and  organ  tissue, 
using  the  scalpel  end  effector.  With  the  grasper,  we  have 
demonstrated  free-needle  threading,  suturing — including 
anastomosis  (surgical  resection)  of  pig  intestine — and  blunt 
dissection. 


Figure  8. 120-Hz  (fleld-rate)  Stereographic 
Display  System 


A.  Experimental  Design 

We  have  also  conducted  preliminary  psychophysical 
experiments  with  different  manipulation  tasks  and  viewing 
conditions  to  compare  telepresence  with  both  free-hand  and 
conventional  laparoscopic  surgical  manipulation.  Three 
subjects  (two  engineers  and  a  surgeon  with  laparoscopic 
training)  were  involved.  Completion  time  was  measured  for; 

•  Bead  Transfer — ^Pick-and-place  transferring  of  0.25-inch 
OD/0.125-inch  ID  beads  (cut  sections  of  plastic  tubing) 
from  pins  spaced  approximately  1.00  inch  apart.  This  is 
a  positioning  task.  Three  beads  were  transferred. 

•  Cannulization — Insertion  of  a  1.00-inch  length  of 
0.125  OD  semi-rigid  plastic  tubing  into  a  slit  in  the  side 
of  0.125  ID  soft  latex  tubing.  This  is  both  a  positioning 
and  orientation  task  that  simulates  surgical  cannulization 
of  a  vessel  or  duct  with  a  catheter. 

Four  manipulation  conditions  were  investigated: 

•  Free-Hand — ^Using  a  hemostat  held  directly  by  the  test 
subject.  Viewing  was  with  the  unaided  eye.  This 
simulates  open  surgery. 

•  Laparoscopic,  2D — ^Using  a  laparoscopic  grasper 
(approx.  12  inches  long)  pivoted  through  a  rubber  sheet 
5  inches  above  the  task  (simulating  the  abdominal  wall). 
Viewing  was  through  a  video  camera  about  50  inches 
away,  zoomed  to  give  a  4-inch  field  of  view  across  the 
monitor  at  the  subject’s  eye  level.  This  simulates 
laparoscopic  surgery,  except  that  it  minimizes  depth  cues 
because  of  the  50-inch  viewing  distance. 

•  Telepresence  System,  3D — ^Using  the  SRI  telepresence 
system  with  the  grasper  end  effector.  Viewing  was 
stereoscopic,  also  zoomed  to  give  a  4-inch  field  of  view 
across  the  monitor.  This  simulates  open  telepresence 
surgery. 

•  Telepresence  System,  2D — ^Using  the  SRI  telepresence 
system,  except  viewed  only  with  the  left  camera. 

B.  Procedure 

Each  subject  started  th^  series  on  a  different 
manipulation  test.  This  balanced  the  effect  of  practice  over 
the  conditions.  The  whole  series  took  less  than  one  hour. 
The  subject  was  allowed  to  practice  the  task  once,  then  was 
timed  on  the  next  three  successive  performances.  If  the 
subject  dropped  a  bead  or  tube,  it  was  replaced  by  the 
experimenter.  Errors  were  not  counted.  A  stopwatch  was 
used  to  measure  the  task  time. 


C.  Results 

The  results  are  shown  in  Figures  9  and  10.  Generally, 
the  performance  of  all  three  subjects  was  similar, 
cannulization  took  about  twice  as  long  as  the  transfer. 
Free-hand  was  the  fastest  condition,  with  telepresence  3D 
taking  about  twice  as  long.  Telepresence  2D  took  about  20% 
longer  than  telepresence  3D.  With  the  laparoscope  2D,  tasks 
took  about  7  times  as  long  for  bead  transfer  and  15  times  as 
long  for  tube  cannulization. 

rV.  PERFORMANCE  EVALUATION: 
TWO-HANDED  OPERATION 

We  have  also  conducted  informal  performance  tests  to 
assess  the  capabilities  of  the  two-handed  telepresence 
system  in  performing  telesurgery.  With  two  hands, 
assistance  in  tensioning  and  positioning  tissue  is  not 
required.  We  have  demonstrated  the  following  capabilities 
with  a  high  level  of  dexterity: 

•  Two-handed  manipulation  using  graspers  to  stretch  and 
dissect  tissue,  locate  and  clamp  a  bleeding  artery,  and 
explore  a  pig  abdomen  (examining  intestine,  liver, 
spleen,  etc.  for  damage  and  remove  foreign  bodies). 
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Figure  9.  Results  of  the  Bead-Transfer  Task 
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Figure  10.  Results  of  the  Cannulization  Task 


•  Unaided  surgical  dissection  in  excised  animal  organ 
tissue,  using  the  scalpel  end  effector  in  one  hand  and  a 
surgical  grasper  in  the  other. 

•  Unaided  surgical  dissection  using  the  micro  scissors  end 
effector  in  one  hand  and  a  surgical  grasper  in  the  other. 

•  Tying  knots  in  suture  materials  using  the  instrument-tie 
technique. 

•  Tying  a  square  knot  in  suture  materials  using  the  normal 
method  with  two  hands.  , 

•  Suturing-including  anastomosis  (surgical  resection)  of 
pig  intestine  using  a  needle  holder  and  a  grasper 
employing  both  individually  tied  sutures  and  running  or 
‘‘purse  string”  sutures. 


We  have  not  yet  conducted  time  and  motion  studies  of 
these  new  capabilities.  Generally,  performance  with  the 
manipulators  is  somewhat  slower  than  with  hand-held 
instruments,  “free-hand  condition.”  Performance  with  the 
manipulators  is  much  faster  than  with  lq)aroscopic  surgery 
techniques.  Laparoscopic  suturing,  which  can  be  only 
haltingly  accomplished  by  the  most  dedicated  surgeons  after 
many  hours  of  practice,  can  be  accomplished  immediately 
by  an  untrained  operator  using  the  telepresence  system. 

V.  CONCLUSIONS 

As  a  result  of  the  care  we  have  taken  to  make  the 
telepresence  system  duplicate  the  operator’s  direct  visual, 
tactile,  and  aural  experience,  we  have  developed  an 
effective  system  remote  manipulation  environment  with  a 
compelling  sense  of  realism.  The  image  of  the  slave  as  seen 
by  the  operator  is  congruent  with  the  master,  and  the  slave 
manipulator  is  seen  to  move  in  the  direction  the  operator 
commands  by  hand  motion. 

As  evidenced  by  these  experimental  results,  the 
telepresence  system  has  proved  much  more  effective  than 
conventional  laparoscopic  technique  and  is  close  to  the 
performance  of  free-hand  (open  surgery).  A  laparoscopic 
version  of  the  telepresence  system  is  under  development  and 
is  expected  to  bring  to  laparoscopic  surgery  the  same 
dexterity  that  it  has  demonstrated  for  remote  open  surgery. 
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BACKGROUND 

The  Telepresence  Surgery  System  developed  by  SRI 
International  has  been  previously  described.[l-6]  It  is 
intended  for  three  distinct  applications:  (1)  telesurgery, 
wherein  the  surgeon  may  participate  in  operations 
performed  at  remote  sites;  (2)  minimally  invasive  surgery, 
especially,  laparoscopic  surgery — telepresence  makes  it 
look  and  feel  like  open  surgery;  and  (3)  microsurgery,  in 
which  the  surgeon's  dexterity  is  enhanced  through  scaling 
of  the  image,  the  motions,  and  the  force  feedback. 

In  this  paper  we  report  on  recent  improvements  to  the 

remote-surgery  version  of  the  system  and  especially  on  its 

adaptation  for  mobile  telesurgery  on  the  battlefield.  . 

Fig.  1 .  The  latest  surgeon's  console  provides  two  hand- 

control  masters  with  surgical-instrument  handles,  a 

stereographic  view  into  the  surgical  site,  and  a 

microphone  and  speakers  for  communicating  with 

the  surgical  assistant  or  nurse  in  the  operating 

room.  Looking  straight  ahead,  the  surgeon  sees  a 

panoramic  view  across  the  remote  surgical  table, 

on  three  side-by-side  LCDs. 


THE  TELEPRESENCE  SURGERY  SYSTEM 

SRI’s  Telepresence  Surgery  System  consists  of  two 
main  modules:  a  surgeon’s  console,  shown  in  Fig.  1,  and  a 
remote  surgical  unit  (RSU),  located  at  the  surgical  table 
and  shown  in  Fig.  2.  The  surgeon  sits  at  the  console  and 
looks  down  into  the  surgical  field,  a  “virtual  workspace” 
recreated  by  a  special  arrangement  of  a  120-field/s 
stereographic  video  monitor  with  a  liquid-crystal  shutter 


Fig.  2.  The  remote  surgical  unit  (RSU)  comprises  two 
slave  manipulators  with  interchangeable  surgical 
instruments,  two  video  cameras  positioned 
correctly  with  respect  to  the  manipulators  and 
canted  in  for  stereo  convergence  in  the  middle  of 
the  operating  field,  microphones,  and  a  trio  of 
video  cameras  (not  shown)  that  produce  a 
panoramic  view  of  the  operating  room  from  the 
surgeon’s  perspective. 

(the  operator  wears  passive  polarized  glasses)  and  a  mirror. 
In  the  open-surgery  version  of  the  system,  described  here, 
the  stereographic  video  is  generated  by  a  pair  of  video 
cameras  on  the  RSU;  they  are  positioned  over  the  patient  in 
special  relationship  to  the  surgical  manipulators,  as 
described  below. 

The  surgeon  reaches  under  the  mirror  and  grasps  two 
surgical-instrument  handles  that  operate  two  master 
manipulators,  each  of  which  directs  the  movement  of  a 
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slave  manipulator  at  the  RSU  by  means  of  a  highly 
responsive,  force-reflecting  servo-controller.  Affixed  to  the 
ends  of  the  manipulator  arms  are  interchangeable  surgical 
instruments — forceps,  needle  drivers,  bowel  graspers, 
scalpels,  and  cautery  tips.  As  shown  in  Fig.  3,  the  surgeon 
sees  the  instruments  in  the  stereographic  image — they 
appear  to  be  emerging  from  the  handles  in  his  hands.  When 
he  moves  his  hand  controls,  the  instruments  move  as  if  they 
were  rigidly  attached  to  the  controls.  When  the  instruments 
touch  the  tissue  or  tug  on  a  suture,  that  force  is  reflected 
back  through  the  instrument  handles  in  a  realistic  manner. 
Sounds  from  the  instruments  and  the  surgical  assistant’s 
voice  are  picked  up  by  stereo  microphones  and  relayed  to 
speakers  in  the  surgeon’s  console.  To  the  surgeon,  it  looks, 
it  feels,  and  it  sounds  as  if  he  is  performing  the  task  directly 
on  the  patient,  with  conventional  surgical  instruments.  The 
strength  of  this  perception  enables  the  surgeon  to  carry  out 
complex  tasks  with  quick,  precise  motions. 

The  masters  and  slaves  are  lightweight,  responsive, 
well-balanced,  and  gravity-compensated.  Friction  has  been 
minimized  by  reducing  the  stages  of  gearing,  using  low- 
friction/low- inertia  actuators,  and  using  low-friction 
bearings  throughout.  Thus,  the  usual  drawbacks  of  inertia 
and  friction — degraded  operator  performance  and  fatigue 
after  prolonged  use — have  been  largely  eliminated.  The 
bilateral  control  system  provides  for  scaled  motion  and 
force  reflection,  which,  along  with  scaled  video,  enable  the 
surgeon  to  perform  microsurgery  with  normal  hand 
motions.  The  current  open-surgery  manipulators  have  five 
degrees  of  freedom,  with  force  feedback  in  each  axis 
(including  opening  and  closing  the  jaws  of  graspers  or 
scissors).  More  dexterous  manipulators  and  a  wider 
assortment  of  easily  interchanged  surgical  instruments  are 
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Fig.  3.  As  illustrated  in  this  superposition,  the  surgeon 

sees  the  instruments  in  the  stereographic  image — 
they  appear  to  be  emerging  from  the  handles  in  his 
hands  and  they  move  in  exact  synchronization  with 
his  hand  movements.  To  the  surgeon,  it  looks,  it 
feels,  and  it  sounds  as  if  he  is  performing  the  task 
directly  on  the  patient,  with  familiar  surgical 
instruments.  Responsive,  force-reflecting  servos 
enable  the  surgeon  to  operate  quickly  and 
confidently. 

being  incorporated  into  a  new  version  of  the  system,  now 
under  development. 

A  laparoscopic  version  of  the  RSU,  not  described 
here,  employs  very  different  remote  manipulators,  designed 


especially  for  operation  through  cannulas.  A  stereo 
laparoscope  is  substituted  for  the  camera  pair. 


REMOTE  SURGERY  ON  THE  BATTLEFIELD 

Potentially,  telepresence  surgery  can  make  a 
significant  contribution  to  the  management  of  combat 
trauma.  Nine  out  of  ten  combat-related  deaths  from 
nonfatal  injuries  occur  in  the  combat  zone  prior  to 
evacuation,  the  majority  from  loss  of  blood.  Many  of  these 
lives  could  be  saved  if  surgical  treatment  were  available 
within  one  hour  of  injury — ^referred  to  as  the  “Golden 
Hour.”  However,  in  modem  warfare,  the  front  line  may 
move  forward  so  quickly  that  mobile  army  surgical 
hospitals  (MASH  units)  cannot  be  established  close  enough 
to  the  combat  zone  to  be  effective.  Currently,  the  only  care 
available  to  the  seriously  wounded  soldier  within  the 
combat  zone  is  that  provided  by  the  field  medic,  whose 
goal  is  to  stabilize  the  victim  and  prepare  him  for 
evacuation.  Sending  combat-ready  trauma  surgeons  into 
the  combat  zone  is  now  being  given  serious  consideration; 
it  is  a  very  unattractive  prospect,  one  which  we  hope  will 
be  made  unnecessary  by  the  advent  of  telepresence 
surgery.[7] 

To  provide  surgical  services  in  the  zone  of  combat, 
SRI  is  developing  a  battlefield  version  of  its  Telepresence 
Surgery  System.  This  version  has  been  installed  in  a  mobile 
surgical  vehicle,  dubbed  the  medical  emergency  forward 
area  surgical  telepresence  (MEDFAST),  envisioned  to  be 
deployed  as  illustrated  in  Fig.  4.  After  first  aid  has  been 
administered  by  the  medic,  the  wounded  soldier  in  need  of 
immediate  surgery  is  brought  to  the  MEDFAST  and  put  on 
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Fig.  4.  From  a  position  safely  behind  the  lines,  the  telepresence  combat  surgeon  of  the  future  will  carry  out  surgeries  in 
mobile  surgical  vehicles  operating  throughout  the  zone  of  combat,  with  the  assistance  of  medics  assigned  to  each 
vehicle.  First  field  tests  will  employ  a  point-to-point  microwave  link  to  carry  video,  servo,  audio,  and  patient 
monitoring  information,  which  eventually  may  be  relayed  by  low-earth-orbit  satellites. 


the  operating  table.  The  telepresence  combat  surgeon 
carries  out  only  a  limited  spectrum  of  procedures.  These 
are  primarily  for  stabilizing  wounded  soldiers,  so  that  they 
can  be  evacuated  when  conditions  permit,  and  for  treating 
conditions  that,  if  deferred,  would  seriously  affect  recovery. 
The  most  threatening  conditions  are  airway  obstructions, 
sucking  chest  wounds,  and  profuse  bleeding.  Working  hand 
in  hand  with  the  medic,  the  surgeon  can  remotely  explore 
for  and  repair  injuries  to  major  vessels  and  abdominal  and 
pelvic  organs,  remove  bone  fragments  and  foreign  material, 
and  debride  damaged  tissues.  The  wound  can  then  be 
packed  and  dressed;  wound  closure  is  left  to  the  next 
echelon  of  care.  Thus,  telepresence  surgery  greatly 
augments  the  trauma  care  that  the  medic  alone  can  provide. 


This  concept  was  first  demonstrated  in  June  1993 
during  a  medical  field  exercise  held  at  Ft.  Gordon  in 
Augusta,  GA.  In  a  tent,  Army  surgeons,  seated  at  an  early 
version  of  our  console  operated  on  an  “injured  soldier” — 
actually,  a  life-like  mannequin  with  pig  intestines  in  its 
abdominal  cavity.  A  medic  at  the  patient’s  side  assisted  the 
surgeon.  The  patient,  the  RSU,  and  the  medic  were  located 
in  a  transportable  operating  room  (OR)  30  meters  from  the 
tent.  Cables  carried  the  control,  video,  and  audio  signals 
between  the  units  (a  high-bandwidth,  two-way  radio  link 
would  be  required  in  an  actual  combat  situation).  At  Ft. 
Gordon,  only  a  single  master-slave  combination  was  used; 
nevertheless,  the  potential  was  evident  and  the  surgeons 
were  enthusiastic. 
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Encouraged  by  the  Ft.  Gordon  results,  we  began 
preparation  for  a  much  more  realistic  demonstration,  in 
which  surgery  would  be  carried  out  in  a  combat-ready 
vehicle  outfitted  as  a  mobile  OR.  The  new  OR  is  in  an 
XM577A3  tracked,  armored  vehicle,  shown  in  Fig.  5, 
which  has  been  equipped  with  a  special  operating  table,  an 
electrically  controlled  gantry  for  positioning  the  RSU  over 
the  patient,  and  patient  monitoring  equipment. 

The  new  surgeon’s  console  (Figs.  1  and  3)  was  constructed 
and  outfitted.[8]  This  version  incorporates  two  hand- 
control  masters;  the  earlier  version  had  only  one.  In 
addition,  it  has  improved  stereographic  video,  a 
microphone  and  speakers  for  communicating  with  the 
medic,  surgical  assistant,  or  nurse  in  the  OR,  and  three 


side-by-side  liquid  crystal  displays  (LCDs)  that  provide  a 
panoramic  view  across  the  operating  table.  The  telesurgeon 
looking  up  from  the  surgical  wound  sees  the  medic’s  face 
and  any  activity  in  the  OR  from  the  same  perspective  he  or 
she  would  have  had  if  standing  at  tableside. 

This  equipment  was  transported  to  Washington,  D.C., 
in  October  1994,  and  demonstrated  at  the  annual 
convention  of  the  Association  of  the  U.S.  Army 
(subsequent  demonstrations  in  Washington  were  made  in 
April  and  May  1995).  Still  linked  by  a  cable,  the  two  units 
were  situated  160  meters  apart — the  577  in  the  hotel 
parking  lot  and  the  surgeon’s  console  in  a  third-floor 
exhibition  hall.  On  the  operating  table  lay  a  soldier 
mannequin  with  lacerated  pig  intestines  and  liver  in  its 


Fig.  5.  Operating  room  in  a  tracked,  armored  vehicle,  (a)  The  XM577A3  Armored  Tactical  Command  and  Control  System 
vehicle,  which  was  modified  for  this  project  by  Foster-Miller  Corp.,  (b)  Interior  of  the  577,  showing  the  operating 
table,  surgical  manipulators,  cameras  and  lights,  the  medic,  and  auxiliary  surgical  and  support  equipment. The 
“patient”  was  a  dummy  with  pig  intestines,  shrapnel,  and  a  simulated  bleeding  artery  in  its  abdominal  cavity. 
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abdominal  cavity,  along  with  bits  of  shrapnel,  and  a 
simulated  bleeding  artery  squirting  blood  (activated 
surreptitiously  by  the  medic).  This  is  representative  of  the 
type  of  abdominal  wound  suffered  in  combat.  [9] 

Surgeons,  and  many  others,  have  now  had  the 
opportunity  to  Ai  at  the  console,  expose  and  examine  the 
injured  organs,  suture  lacerations,  remove  shrapnel,  and 
expose  the  “bleeder,”  which  the  medic  then  clamped. 

Surgeon  and  medic  worked  together  as  if  they  were 
standing  across  the  table  from  each  other.  For  example,  the 
medic  would  pass  the  suture  to  the  surgeon  and  then 
provide  countertension  on  the  tissues  while  the  surgeon 
sewed.  Surgeons  were  able  to  run  suture  lines  and  do 
instrument  ties  with  high  dexterity  and  speed. 

The  general  consensus  was  that  the  feasibility  of 
telepresence  surgery  on  the  battlefield  had  been 
demonstrated,  although  a  number  of  improvements  were 
needed.  Preparation  has  begun  for  actual  field  trials  of  the 
system.  Three  notable  features  are  being  added:  (1)  a  pair 
of  seven-degree-of-freedom  hand  controls  and 
manipulators,  (2)  a  broader  array  of  surgeon- 
interchangeable  instruments,  and  (3)  a  wireless 
communication  system. 

COMMENTS  AND  CONCLUSIONS 

SRI  has  established  that  precise  surgical  procedures 
can  be  carried  out  with  telepresence.  We  believe  that  within 
the  next  decade  telepresence  surgery  will  become  an 
established  component  of  a  new,  technologically  enhanced 
mode  of  cost-effective  health  care  delivery.  Because  of  the 
substantial  medical  and  financial  benefit  that  telepresence 


will  bring  to  laparoscopic  and  other  minimally  invasive 
procedures,  its  first  routine  use  is  expected  to  be  within  the 
individual  hospital  and  surgical  center.  Subsequently, 
remote  surgery  also  will  become  practical.  We  have 
demonstrated  telepresence  surgical  procedures,  including  a 
wide  variety  of  procedures  in  live- animal  studies  [10],  over 
160  meters  of  copper  wire.  We  are  now  working  toward 
demonstrating  animal  surgeries  over  a  much  greater 
distance  using  a  microwave  link.  Highly  reliable  fiber  optic 
links  will  be  readily  available  for  civilian  telesurgery. 
Mobile  telepresence  surgical  equipment  may  prove 
especially  effective  for  trauma  care,  not  only  under  combat 
conditions  but  for  civilian  disaster  intervention  as  well. 
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ABSTRACT 


To  assess  the  capabilities  of  our  fully  functional,  prototype  telepresence  surgery 
system,  experienced  surgeons  performed  complete  operative  procedures  on  live, 
anesthetized  pigs.  Cholecystectomy,  the  prototypical  procedure  for  evaluating  the 
integration  of  surgical  skills,  was  successfully  performed  in  six  animals.  There  were  no 
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aborted  attempts  or  complications.  Other  procedures  completed  included  gastrotomy  and 
enterotomy  closures,  anastomosis  of  the  small  intestine,  and  nephrectomy.  No  specific 
training  was  required  for  using  the  telepresence  surgery  system,  and  the  ‘‘feel”  of  the 
system  was  described  as  intuitive.  Operative  times  were  longer  than  required  in 
conventional,  open  surgery,  most  likely  the  result  of  the  four  degrees  of  freedom 
available  in  the  manipulators  of  the  current-generation  system.  Force  feedback  and  high- 
resolution,  stereoscopic  video  input  facilitated  performance.  Surgeons  operating  through 
a  first-generation  telepresence  system  can  achieve  technical  results  equivalent  to  those 
obtained  in  conventional  surgery. 


BACKGROUND 


Telepresence  surgery  is  based  on  the  premise  that  an  immersive  sensory 
environment  can  substitute  for  the  physical  presence  of  a  patient.  ^  It  presumes  that  high- 
fidelity  visual,  auditory,  and  tactile  input  can  be  electronically  transmitted  to  a  surgeon  in 
such  a  way  that  he  or  she  can  use  that  input  to  make  the  same  decisions  that  would  be 
made  in  an  operating  room.  Furthermore,  telepresence  surgery  requires  the  accurate 
replication  of  a  surgeon’s  hand  motions  at  the  remote  site  where  the  patient  is  located. 

The  basic  concepts  of  telepresence  surgery  have  been  demonstrated  with  a 
prototype  telepresence  surgery  system  developed  by  SRI  International.^’^  In  tests  on 
inanimate  models  and  isolated  tissues,  tissue  incision,  cannulation,  suturing,  and  knot 
tying  were  all  successfully  completed.  Tests  of  these  psychomotor  performance  skills, 
although  indicative  of  the  precision  and  accuracy  that  are  possible  with  the  prototype 
system,  do  not  adequately  assess  whether  a  complete  operative  procedure  could  be 
performed  by  telepresence  surgery. 

The  operative  phase  of  surgery  requires  tissue  incision,  retraction  and  exposure, 
dissection,  cutting,  grasping,  suturing,  and  knot  tying.  The  sequence  of  steps  required  for 
a  procedure  is  usually  the  same  from  one  patient  to  the  next,  but  each  individual  has 
unique  anatomic  characteristics.  Further  variability  is  introduced  by  disease  processes 
that  cause  tissue  distortion.  Thus,  real-time  feedback  of  visual  and  tactile  information  to 
the  operating  surgeon  is  critical  for  the  safe  and  accurate  performance  of  surgery. 
Consequently,  to  assess  the  actual  potential  of  telepresence  surgery,  we  performed 
complete  operations  on  live,  anesthetized  swine. 
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METHODS 


The  telepresence  surgery  system  used  in  these  studies  has  been  previously 
described.  A  surgeon  sits  at  a  console  equipped  with  stereoscopic  video  input,  and 
controls  master  manipulators  that  provide  force  feedback.  The  remote  surgical  unit 
where  the  patient  is  located  has  slave  manipulators  with  five  degrees  of  freedom, 
including  instrument  opening  and  closing.  Standard  surgical  instruments  are  available  to 
the  surgeon  (Figure  1).  For  the  experiments  reported  in  this  study,  the  surgeon’s  console 
and  remote  surgical  unit  were  located  in  the  same  surgical  suite,  connected  by  electronic 
cables. 


The  surgeon’s  perception  is  one  of  presence  at  the  operative  field,  providing  the 
potential  to  perform  procedures  with  the  precision  and  accuracy  obtained  if  the  surgeon 
were  physically  present  at  the  operating  table. 

All  animal  care  complied  with  the  Principles  of  Laboratory  Animal  Care 
(formulated  by  the  National  Society  for  Medical  Research)  and  Guide  for  the  Care  and 
Use  of  Laboratory  Animals  (NIH  Publication  No.  86-23,  revised  1985).  Female 
Hampshire  swine  (n=4)  were  anesthetized  by  standard  techniques  and  monitored 
throughout  the  course  of  the  experiments.  At  the  completion  of  the  procedures,  the 
anesthetized  animals  were  euthanized  with  an  overdose  of  barbiturates, 

A  midline  laparotomy  was  performed  and  retractors  placed.  All  subsequent 
procedures  were  performed  using  the  telepresence  surgery  system.  A  fully  trained 
general  surgeon  operated  from  the  control  console,  assisted  by  a  nonphysician  technician 
at  the  remote  location  with  the  animal.  All  procedures  initiated  were  successfully 
completed  using  the  telepresence  surgery  system,  including  cholecystectomy,  gastrotomy 
and  enterotomy  closures  (to  simulated  stomach  and  intestinal  injuries,  respectively), 
repair  of  liver  laceration,  and  nephrectomy.  The  infrarenal  abdominal  aorta  was  exposed 
and  a  1-cm  segment  was  excised.  An  interposition  graft  of  6  mm  polytetrafluoroethylene 
(PTFE)  (Gore-Tex,  a  product  of  WL  Gore  &  Associates,  Inc.,  Flagstaff,  AZ)  was  placed. 
In  addition,  the  telepresence  surgery  system  was  used  to  incise  groin  skin,  perform 
dissection  and  exposure  of  the  common  femoral  artery,  and  isolate  the  vessels.  A  3-cm 
long  arteriotomy,  simulating  arterial  injury,  was  closed  by  means  of  the  telepresence 
surgery  system."* 

Before  the  creation  of  liver  lacerations  or  simulated  vascular  injuries,  the 
animals  were  systematically  anticoagulated  with  sodium  heparin,  100  U/kg,  administered 
intravenously. 

Performance  measures  included  time  to  task  completion,  quality  of  the 
completed  procedure,  and  system  limitations  noted.  For  arterial  repairs,  vessel  patency 
and  suture  line  integrity  were  assessed. 


Figure  la.  Slave  manipulator  arm  equipped  with  a  needle  holder 
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Figure  1b.  Rartge  of  surgical  instruments  available  for  telepresence  surgery 
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RESULTS 


The  surgeons  who  used  the  telepresence  surgery  system  received  no  special 
training,  and,  after  a  short  practice  period  on  ex  vivo  tissues,  were  able  to  use  the  system 
successfully  to  operate  on  live  animals.  Tissue  manipulation,  dissection,  incision,  and 
suturing  were  all  performed  without  observable  errors.  In  cholecystectomies,  cystic  ducts 
and  arteries  were  readily  identified,  isolated,  and  ligated.  Gallbladders  were  dissected 
free  from  the  liver  without  damage  to  either  organ.  Gastrotomy  closures,  performed  in 
two  layers,  and  inverting  enterotomy  closures  appeared  identical  to  procedures  in  which 
conventional,  open  surgical  techniques  were  used. 

Liver  lacerations  were  closed  with  mattressed  sutures  or  with  advanced  fibrin 
hemostatic  bandages.  Both  methods  controlled  steady  bleeding,  without  causing  further 
tissue  damage.  Patency  of  vessels  was  maintained  after  arterial  repairs,  and  no 
hemorrhage  was  noted  from  suture  lines  (Figure  2). 

The  times  required  for  procedures  performed  with  the  telepresence  surgery 
system  were  two  to  three  times  longer  than  those  performed  with  conventional  operative 
techniques.  Increased  times  were  required  for  tissue  dissection,  suturing,  and  knot  tying, 
with  no  one  component  predominating  in  length.  Prolonged  use  of  the  system  resulted  in 
only  minimal  decrease  in  task  performance  times. 


DISCUSSION 


The  obvious  uses  of  telepresence  surgery  are  in  extending  the  expertise  of  a 
surgeon  to  locations  where  that  is  not  currently  possible.  On  the  battlefield,  casualties  die 
from  exsanguinating  hemorrhage.  By  using  a  mobile  military  vehicle  equipped  with  a 
remote  surgical  unit  and  nonphysician  medical  assistant,  a  surgeon  operating  from  a  field 
hospital  30  to  70  miles  away  could  perform  life-saving,  stabilizing  procedures  that  would 
allow  casualties  to  be  safely  evacuated  for  definitive  care.^  Basic  procedures  performed 
in  trauma  surgery  could  be  successfully  performed  with  the  telepresence  system. 

The  increased  times  required  for  task  completion  with  the  telepresence  surgery 
system  were  consistent,  and  most  likely  resulted  from  the  constraints  imposed  by  having 
only  four  degrees  of  freedom  in  the  manipulator  arms,  whose  range  of  motion  simulates  a 


Figure  2.  PTFE  aortic  interposition  graft  replacing  excised  segment  of  infrarenal  aorta 
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human  upper  extremity  with  a  nonmobile  (fused)  wrist.  This  limitation  was  most 
apparent  to  surgeons  when  dissecting  or  suturing  tissues  on  an  axis  parallel  to  the  remote 
surgical  unit’s  orientation.  Although  the  feasibility  of  telepresence  surgery  was  apparent 
despite  this  limitation,  improved  wrist  functions  will  be  required  for  clinical  acceptance. 
The  next-generation  system,  now  being  assembled  at  SRI,  will  provide  a  full  seven 
degrees  of  freedom  in  the  manipulator  arms.  With  this  improvement,  a  surgeon  will  be 
able  to  operate  freely  about  all  axes  of  rotation. 

Surgeons  readily  adapt  to  performing  under  adverse  conditions.  Laparoscopic 
surgery  reduces  the  tactile  feedback  available  to  a  surgeon,  removes  stereoscopic  visual 
cues,  and  introduces  an  abnormal  instrument  fulcrum  at  the  anterior  abdominal  wall.  Yet 
in  a  short  period  of  time,  minimally  invasive  techniques  have  been  adapted  to  the 
performance  of  many  gastrointestinal  procedures.^  Similarly,  surgeons  can  achieve  good 
results  under  adverse  lighting  and  exposure.  Therefore,  to  accurately  assess  the 
performance  of  a  telepresence  surgery  system,  it  is  not  enough  to  conduct  time  and 
motion  studies  in  isolated  ex  vivo  tasks  on  inanimate  objects.  Surgical  procedures 
requiring  the  real-time  assimilation  of  information  and  integration  of  component  tasks 
will  be  the  benchmarks  for  development  of  system  specifications. 

Mobile  telepresence  surgery  will  require  wireless  microwave  links  between  the 
surgeon’s  console  and  the  remote  surgical  unit.’  Key  performance  issues  related  to  data 
packaging  and  transmission  must  be  addressed.  Specifically,  the  image  resolution  needed 
for  a  surgeon  to  accurately  perform  surgery  will  determine  the  bandwidth  requirements 
and  the  potential  for  image  compression.  In  addition,  the  lag  times  in  instrument 
response  that  are  acceptable  to  a  surgeon  must  be  determined. 

The  role  of  telepresence  surgery  in  the  operating  environment  of  the  future 
remains  to  be  established.  On  the  basis  of  our  experience  with  the  first  complete 
prototype  system  available,  however,  we  conclude  that  telepresence  technology  clearly 
has  the  potential  to  revolutionize  the  practice  of  surgery. 
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ABSTRACT 

SRI  International  is  currently  developing  a  prototype  remote  telepresence 
surgeiy  system,  for  the  Advanced  Research  Projects  Agency  (ARP A),  that  will  bring  life¬ 
saving  surgical  care  to  wounded  soldiers  in  the  zone  of  combat.  Remote  surgery  also  has 
potentially  important  applications  in  civilian  medicine.  In  addition,  telepresence  will  find 
wide  medical  use  in  local  surgery,  in  endoscopic,  laparoscopic,  and  microsurgery 
applications.  Key  elements  of  the  telepresence  technology  now  being  developed  for 
ARPA,  including  the  telepresence  surgeon’s  workstation  (TSW)  and  associated  servo 
control  systems,  will  have  direct  application  to  these  areas  of  minimally  invasive  surgery. 
The  TSW  technology  will  also  find  use  in  surgical  training,  where  it  will  provide  an 
immersive  visual  and  haptic  interface  for  interaction  with  computer-based  anatomical 
models.  In  this  paper,  we  discuss  our  ongoing  development  of  the  MEDFAST 
telesurgery  system,  focusing  on  the  TSW  man-machine  interface  and  its  associated  servo 
control  electronics. 


INTRODUCTION 

SRI  International  is  currently  developing  a  prototype  remote  telepresence 
surgery  system,  for  the  Advanced  Research  Projects  Agency  (ARPA),  that  will  bring  life¬ 
saving  surgical  care  to  critically  wounded  soldiers  in  the  combat  zone  (CZ).  Using  SRI’s 
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telepresence  technology,  a  surgeon  located  in  a  mobile  army  surgical  hospital  (MASH) 
unit  or  base  hospital  will  carry  out  emergency  surgical  procedures  on  soldiers  in  an 
armored  mobile  surgical  vehicle  in  the  CZ.  Dubbed  the  Medical  Emergency  Forward 
Area  Surgical  Telepresence  (MEDFAST)  system,  this  vehicle  will  contain  a  fiilly 
equipped  telepresence  operating  theater,  and  a  medic  in  the  MEDFAST  will  assist  the 
remote  surgeon. 

Remote  telepresence  surgery  also  has  applications  in  civilian  medicine,  where  it 
will  enable  specialists  in  regional  medical  centers  to  treat  patients  in  outlying  clinics.  In 
addition,  telepresence  will  most  likely  find  wide  medical  use  in  local  surgery — in 
endoscopic,  laparoscopic,  and  microsurgeiy  applications,  and  will  play  an  important  role 
in  surgical  training,  providing  an  immersive  visual  and  haptic  interface  to  computer- 
based  anatomical  models. 

In  current  practice,  microsurgery  involves  delicate  manipulation  of  small  tissue 
structures  viewed  through  a  stereo  microscope.  While  major  reconstructive  efforts  such 
as  reattachment  of  severed  digits  are  routinely  performed,  these  microsurgical  procedures 
are  often  long  and  exhausting,  with  the  surgeon  spending  many  hours  of  concentrated 
work  looking  through  microscope  eyepieces,  sitting  in  an  awkward  position  [Franken  et 
al.,  1995].  In  addition,  human  performance  limitations  preclude  the  extension  of 
microsurgical  techniques  to  new  applications  using  higher  magnification.  The  time  to 
complete  a  task  increases  in  proportion  to  the  visual  magnification  required  to  complete  it 
[Szabo,  1994].  Therefore,  increases  in  magnification  result  in  longer  procedures,  which 
increase  patient  risk  and  incur  greater  financial  costs. 

Similar  limitations  exist  for  laparoscopic  surgery  as  currently  practiced.  It  has 
largely  replaced  open  surgery  for  relatively  simple  procedures  such  as  cholecystectomy 
and  hernia  repair,  and  brings  substantial  benefits  to  the  patient,  including  shorter  hospital 
stay,  faster  recovery,  and  less  cosmetic  damage.  However,  with  current  instrumentation, 
laparoscopic  procedures  require  more  operating  room  time  and  are  much  more  difficult 
for  the  surgeon  than  equivalent  open  procedures.  As  a  result,  relatively  few  surgeons  will 
be  able  to  progress  from  laparoscopic  cholecystectomy  to  more  complex  procedures  in 
the  bowel  and  esophagus.  The  problem:  laparoscopic  surgery  currently  entails 
awkwardly  maneuvering  long  instruments  that  are  ftilcrumed  in  the  patient’s  abdominal 
wall  (so  that  the  motion  of  the  tool  tips  is  opposite  to  the  motions  of  the  surgeon’s  hands), 
while  viewing  the  surgical  field  on  a  2D  video  display  located  on  a  stand  above  the 
patient.  This  arrangement  makes  hand-eye  coordination  very  difficult,  and  provides  the 
surgeon  with  very  little  force  feedback. 

SRI’s  telepresence  technology  can  overcome  many  of  the  limitations  imposed 
by  current  minimally  invasive  surgery  (MIS)  instrumentation  and  methods.  By  enabling 
a  great  increase  in  the  range  and  number  of  procedures  that  can  be  performed, 
telepresence  will  bring  the  benefits  of  MIS  to  many  more  patients. 

In  the  case  of  microsurgery,  telepresence  systems  will  provide  motion  and  visual 
scaling  along  with  amplified  force  feedback  that  will  allow  surgeons  to  perform 
microsurgical  procedures  with  the  same  speed,  skill,  and  dexterity  they  achieve  in  full- 
scale  open  surgery.  Similarly,  for  laparoscopic  and  other  endoscopic  procedures, 
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telepresence  systems  will  provide  an  intuitive  and  effective  interface  for  telepresence 
surgery  that  will  greatly  enhance  surgeons’  capabilities. 

Key  elements  of  the  MEDFAST  system  we  are  now  developing,  including  the 
telepresence  surgeon’s  workstation  (TSW)  and  associated  servo  electronics,  will  find 
direct  use  in  these  MIS  telepresence  systems  and  will  play  an  important  role  in  future 
MIS  practice.  In  addition,  the  new  TSW  will  be  useful  in  surgical  training,  providing  an 
excellent  visual  and  haptic  interface  for  interaction  with  computer-based  anatomical 
models. 


In  this  paper  we  discuss  our  ongoing  development  of  the  MEDFAST  system, 
focusing  on  the  TSW  man-machine  interface  and  its  associated  servo  control  electronics. 


BACKGROUND 

SRI  International  has  previously  developed  and  demonstrated  a  prototype 
telepresence  surgery  system  [Green  et  al.,  1995]  that  has  been  used  to  perform  successful 
remote  surgical  procedures  on  animals.  Other  robotic  and  telerobotic  systems  have  been 
developed  for  surgical  applications  [Burckhardt  et  al.,  1995;  Taylor  et  al.,  1994],  but  only 
the  SRI  system  provides  the  combination  of  responsive,  force-reflecting  telemanipulators 
and  integrated  visual  feedback  that  allows  surgeons  to  remotely  perform  the  full  spectrum 
of  surgical  maneuvers — for  example,  cutting,  suturing,  dissecting — ^with  a  high  degree  of 
dexterity  and  effectiveness  comparable  to  their  hands-on  skills. 

The  SRI  telepresence  technology  is  an  entirely  new  method  of  remote 
manipulation,  with  important  potential  applications  in  remote  surgery  and  MIS. 

Surgeons  who  have  worked  with  SRI’s  telepresence  surgeiy  demonstration  system,  with 
its  fully  integrated  combination  of  three-dimensional  (3D)  video  imaging,  precise  remote 
manipulation  with  handheld  surgical  instruments,  and  force  feedback,  agree  that  it  is  so 
compellingly  realistic  as  to  immediately  dispel  any  sense  of  remote  operation.  Using  the 
demonstration  system,  these  surgeons  have  successfully  performed  three  series  of  remote 
surgical  procedures  on  pigs  [Bowersox  et  al.,  1996].  These  first-ever  remote  surgeries 
included  treatment  of  a  variety  of  simulated  abdominal  and  vascular  injuries,  including 
cholecystectomy,  nephrectomy,  repair  of  gastrotomy  and  enterotomy,  and  repair  of 
femoral  artery,  liver,  and  bladder  lacerations. 

Shown  in  Figure  1,  the  demonstration  system  consists  of  two  modules:  a  work¬ 
site  module,  where  the  actual  object  manipulation  takes  place,  and  an  operator  module 
that  contains  a  strikingly  realistic,  3D,  video  reproduction  of  the  actual  work  space  (not  a 
computer  model).  This  demonstration  system  incorporates  many  advanced  technologies, 
including  a  highly  responsive,  force-reflecting  master/slave  manipulator  with  4  degrees 
of  freedom  (DOF),  stereoscopic  video,  and  stereophonic  sound.  In  effect,  the  operator 
grasps  surgical  instruments,  reaches  into  what  appears  to  be  the  actual  workspace,  and 
carries  out  complex  tasks  with  quick,  sure  motions.  Operating  surgeons  feel  as  though 
they  are  performing  tasks  right  before  their  own  eyes.  The  3D  image  is  created  by  a 
stereographic  video  system,  viewed  with  a  mirror.  The  surgeon  looks  downward, 
through  the  mirror,  to  see  the  surgical  site  below  it,  just  where  he  or  she  is  reaching 
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(a)  The  telepresence  surgeon’s 

workstation  of  the  demonstration 
system  provides  two  hand-control 
masters  with  surgical  instrument 
handles*  a  stereographic  view  of 
the  surgical  field,  and  a 
microphone  and  speakers  for 
communicating  with  the  assistant 
at  the  remote  site. 
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(b)  The  demonstration  system  remote 
surgery  unit  comprises  two  slave 
manipulators  with  interchangeable 
surgical  instruments,  a  pair 
of  stereographic  video  cameras, 
and  stereo  microphones. 


Figure  1.  Demonstration  telepresence  surgery  system 

with  the  hand  controls.  The  control  is  a  surgical-instrument  handle  mounted  on  a  light, 
well-balanced,  force-reflecting  servo  manipulator— the  master.  An  identical  slave  servo 
manipulator  at  the  operating  table  controls  the  actual  surgical  instrument — scalpel, 
forceps,  and  so  forth — the  tip  of  which  is  seen  in  the  image,  as  if  it  were  actually  attached 
to  the  handle.  When  the  tip  touches  the  tissue,  the  resistance  is  felt  from  the  handle.  The 
overall  effect  is  very  compelling.  With  support  from  the  National  Institutes  of  Health, 
ARPA,  and  other  sources,  SRI  has  continued  the  development  of  this  technology  for 
remote  surgery  as  well  as  laparoscopic  and  microsurgery  applications. 
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THE  ADVANCED  SURGERY  SYSTEM 


System  Block  Diagram 


•  A  simplified  block  diagram  of  the  advanced  system  is  shown  in  Figure  2. 
We  have  established  specifications  for  the  master  and  slave 
telemanipulators  and  the  stereo  video  display.  These  specifications,  which 
form  the  basis  for  the  TSW  design,  are  summarized  below. 


Telepresence  Surgeon's 
Workstation  (TSW) 


•  Two  6-cJof  Hand  Controls  with 
Interchangeable  Handles 

•  Stereographic  Display 

•  Panoramic  Scene  Display 

•  Auxiliary  Controls 

•  Computer  Workstation 

•  Servo  Controllers 

•  Stereo  Video  Processor 

•  Video  Codecs 

•  Audio  Equipment 


Digital 

Microwave 

Communication 

Link 


Remote  Surgery 
Unit  (RSU) 


•  Positionable  Pod 

•  Two  6-dof  Manipulators  with 
Interchangeable  Instruments 

•  Stereographic  Video 
Cameras 

•  Three  Scene  Cameras 

•  Computer  Workstation 

•  Servo  Controllers 

•  Video  Codecs 

•  Audio  Equipment 


Figure  2.  MEDFAST  system  block  diagram.  The  MEDFAST  system  consists  of  the 
telepresence  surgeon’s  workstation  (TSW),  the  communication  Imk,  the 
■  remote  surgery  unit  (RSU),  and  electronics  modules  associated  with  the  TSW 
and  RSU. 


•  Control  handles  for  master  manipulators,  interchangeable  for  surgeon’s  use 
with  either  right  or  left  hand 

-  Hemostat  handle 

-  Forceps  (pickup)  handle 

•  Stereo  vision  system 

-  Display  resolution:  640  X  480  pixels,  minimum 

-  Camera  resolution:  700  horizontal  by  494  vertical  lines 

-  Stereo  display  type:  120  Hz  full-frame  with  active  or  passive  glasses 
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The  New  TSW 

Based  on  the  above  specifications,  we  are  developing  a  new  TSW  with  two 
6-DOF  (plus  grip)  hand  controls,  enhanced  displays,  auxiliary  controls,  and  all-new, 
ruggedized  electronics.  Using  wood  and  metal  mock-ups,  we  have  had  consulting 
surgeons  perform  simulated  procedures,  including  suturing  organ  models,  to  evaluate  the 
range  of  motions  required  for  surgical  procedures  and  guide  our  designs  as  they  evolve. 
The  surgeons  are  experienced  in  trauma,  general,  and  vascular  surgery,  as  well  as  other 
surgical  specialties. 

Master  Manipulator  Design ;  The  proposed  manipulator  design  is  shown  in 

Figure  3. 


(a)  Overview  of  (b)  Detail  of  master  with 

manipulator  pickup  handle 

Figure  3.  Design  of  surgical  manipulators 

TSW  Console  Human  Factors:  The  new  surgeon’s  console  has  been  designed 
to  accommodate  surgeons  with  a  wide  range  of  physical  statures.  The  design  will 
comfortably  seat  men  and  women  ranging  from  the  5th  to  the  95th  percentile  in  height, 
and  will  enable  effective  telepresence  for  all  of  them.  Figure  4,  based  on  data  from 
Woodson  and  Conover  [1964],  illustrates  the  latitude  of  adjustment  necessary.  Our  plan 
is  to  have  the  surgeon  sit  on  a  chair  of  adjustable  height,  which  will  bring  his  or  her  eyes 
to  the  proper  location  to  view  the  display.  A  foot  rest  with  foot  controls  will  be  brought 
up  to  match  a  comfortable  foot  height,  and  (if  necessary)  the  master  arms  can  be  adjusted 
up  or  down  to  a  comfortable  position. 

To  accommodate  the  6-DOF  manipulators,  the  new  surgeon’s  console  will  need 
to  provide  a  much  larger  workspace  than  is  afforded  by  the  current  console.  Accordingly, 
the  new  console  is  designed  to  allow  unimpeded  motion  of  the  control  handles 
throughout  a  range  of  at  least  1  cubic  foot  directly  in  front  of  the  surgeon,  to  allow  rapid 
and  effective  surgical  treatment  of  trauma.  The  concepmal  design  is  shown  in  Figures  5a, 
5b,  and  5c.  For  finer,  more  controlled  procedures,  such  as  suturing  and  micro  dissecting, 
the  console  will  include  positionable  forearm  rests  for  stabilization. 
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Figure  4.  Variation  in  human  dimensions 
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(a)  Perspective  view 


(b)  Side  view 


(c)  Top  view 

Figure  5.  Design  of  telepresence  surgeon’s  workstation 

Both  the  stereographic  display  and  heads-up  display  are  also  included  in  the 
surgeon’s  workstation.  The  stereo  display  is  viewed  through  a  mirror  as  in  our  initial 
telepresence  demonstration  system.  The  heads-up  display  (not  shown  in  Figure  5)  will  be 
at  the  back  of  the  TSW  console.  Additional  items  not  shown  include  foot  controls,  audio 
microphone,  and  stereo  speakers. 
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Servo  Controller 

New  servo  electronics  and  algorithms  have  been  developed  to  control  the 
6-DOF  master  and  slave  manipulators.  To  enable  enhanced  tactile  feedback,  enhanced 
dynamic  response,  and  greater  manipulator  output  force,  we  have  developed  and  tested 
new  control  algorithms  that  employ  a  combination  of  force  and  position  feedback.  The 
algorithms  have  been  tested  using  simulation  and  actual  operation  on  a  single-axis  test 
stand. 


Simulation :  We  have  developed  a  model  of  our  DSP-based  controller  and  a 
motor-driven  test  axis  using  Matlab  and  SIMULINK  (products  of  Math  Works,  Inc.)  to 
aid  in  developing  and  testing  different  bilateral  servo  algorithms.  We  constructed  a 
detailed  analytical  and  numerical  model  of  our  control  hardware  and  the  dynamics  of  the 
servo  control  test  stand  so  as  to  better  understand  the  control  issues  and  to  speed  and 
facilitate  alternative  control  designs.  The  model  includes  discrete  sampling  effects  and 
real-time  computation  delay  effects. 

Servo  Test  Stand :  We  have  built  a  single-axis  servo  test  stand  to  verify  the 
simulation  results.  The  test  stand  consists  of  two  motor-driven  axes  with  gearing  and 
inertia  similar  to  the  elbow  axis  of  the  new  manipulators.  Each  axis  has  a  tachometer  and 
potentiometer  for  determining  step  and  frequency  response.  In  addition,  an  external  drive 
motor  on  one  axis  produces  sinusoidal  input  (simulating  operator  hand  motion)  at 
different  frequencies  for  determining  transfer  functions,  A  force  sensor  on  the  tip  of  each 
arm,  sensitive  to  ±2  lb,  is  interfaced  to  the  analog  inputs  of  our  servo  controller. 

Evaluation  of  Control  Laws :  We  have  evaluated  the  following  known  control 
laws  for  bilateral,  force-reflecting  control  of  teleoperated  manipulators. 

•  Position/position  control :  First  introduced  over  thirty  years  ago  [Burnett, 
1957],  position/position  control  used  bilateral  position  tracking  to  introduce 
some  force  reflection  into  a  teleoperated  manipulator. 

•  Power  steering  and  position/position  control :  Local  power  steering  can 
be  introduced  into  the  position/position  control  scheme  by  feeding  back  a 
force  measurement  to  the  local  controller. 

•  Position/position  control  and  cross-coupled  force  reflection  :  Power 
steering  is  made  symmetric  by  passing  force  measurements  between  master 
and  slave. 

•  Position  forward/force  back:  This  is  an  asymmetric  control  objective  that 
has  the  slave  manipulator  follow  the  master  manipulator’s  position  only, 
while  the  master  manipulator  follows  the  slave’s  force  measurement  only 
[Flateau,  1969].  Additional  damping  is  provided  by  some  velocity 
feedback.  This  approach  can  an  be  used  with  a  single  (slave)  force  sensor. 

•  Ideal  kinesthetic  coupling :  A  physically  derived  control  law  using 
acceleration  feed  forward  and  cross-coupled  force  measurements  provides 
“ideal  kinesthetic  coupling”  [Yokokohji  and  Yoshikawa,  1994]. 
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We  are  currently  evaluating  each  of  the  control  laws  listed  in  Table  1  with  computer 
models  and  the  single-axis  servo  test  stand. 

Table  1 

KINESTHETIC  COUPLING  CONTROL  LAWS 


Performance 

Passive 

Optimality  and  Tuning 

Bilateral  position 
tracking  (BPT) 

Tracks  position,  force 
tracking  contaminated 
by  manipulator  inertia 

Yes 

No  automatic  tuning, 
suboptimal  performance 

Power  steering 
with  BPT 

Reduced  inertia  compared 
to  BPT  alone,  loss  of 
contact  stiffness 

No 

No  automatic  tuning, 
suboptimal  performance 

Bilateral  force 
amplification  with 
BPT 

Good  stiffness  and  low 
inertia 

No 

No  automatic  tuning, 
suboptimal  performance 

Position  forward/ 
force  back 

Good  stiffness  and  low 
inertia,  can  be  used  with 
single  force  sensor 

No 

No  automatic  tuning, 
suboptimal  performance 

Ideal  kinesthetic 
coupling 

Good  stiffness  and  low 
Inertia 

Yes 

Model-based  tuning, 
suboptimal  performance 
(acceleration  filtering) 

New  Control  Electronics :  New  control  electronics  have  been  designed  and' 
built  to  accommodate  both  force-  and  position-feedback  signals. 


COMMENTS  AND  CONCLUSIONS 

As  part  of  SRI’s  ongoing  project  to  develop  a  MEDFAST  telesurgery  system  for 
ARPA,  we  are  developing  an  advanced  TSW  design  that  will  enable  highly  effective 
telepresence  for  remote  surgery  as  well  as  MIS  applications,  and  will  also  serve  as  an 
excellent  immersive  interface  for  surgical  training  using  computer-based  anatomical 
models.  The  TSW  will  include  6-DOF  (plus  grip)  force-reflecting  master  manipulators, 
high-resolution  3D  video,  heads-up  peripheral  video  displays,  foot  switches,  and  other 
I/O  devices.  The  new  console  will  allow  unimpeded  motion  of  the  master  control 
handles  throughout  a  range  of  at  least  1  cubic  foot  directly  in  front  of  the  surgeon, 
enabling  rapid  and  effective  surgical  treatment 
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Telepresence  Technology  in  Medicine: 
Principles  and  Applications 


JOHN  W.  HILL,  MEMBER,  IEEE,  AND  JOEL  F.  JENSEN 
Invited  Paper 


Telepresence  systems  can  improve  surgeons*  performance  in 
minimally  invasive  surgery  (MIS)  and  microsurgery  and  also  en¬ 
able  them  to  operate  on  patients  remotely  over  great  distances. 
In  MIS,  telepresence  technology  allows  surgeons  to  experience 
surgery  as  if  their  hands  and  eyes  were  effectively  inside  the 
patient's  closed  abdomen,  enabling  them  to  work  with  improved 
skill  and  dexterity.  In  microsurgery,  the  technology  can  scale  down 
the  surgeons*  motions,  forces,  and  field  of  view,  allowing  them  to 
skillfully  operate  on  microscopic  anatomy  with  relative  ease.  Our 
systems  also  enable  surgeons  to  treat  patients  remotely  in  inac¬ 
cessible  or  hazardous  locations  with  great  effectiveness,  allowing 
them  to  operate  as  if  they  were  present  at  the  remote  site.  The 
means  for  conveying  human  presence  in  such  systems  is  through 
force-reflecting  manipulators  with  digital  servo  controllers,  stereo 
viewing  systems,  and  communication  links.  Depending  on  the 
application,  the  surgeon  at  the  telepresence  workstation  may  be 
across  the  room  from  the  patient  or  across  the  state,  connected  by 
a  microwave  link  or  communication  network. 

Keywords — Dexterous  manipulators,  micromanipulator,  mini¬ 
mally  invasive  surgery,  surgery,  surgical  simulation,  telepresence, 
telerobotics,  virtual  reality. 


1.  Introduction  and  Background 
A.  Teleoperation 

Teleoperation  systems  have  made  it  possible  for  man  to 
reprocess  plutonium,  service  satellites  in  space,  and  perform 
deep-sea  salvage  operations  from  a  safe  control  room  by 
using  remote  manipulators  and  communication  links.  These 
systems  are  typically  operated  by  teams  of  engineers  acting 
on  data  from  walls  full  of  computer  displays.  Remote  work 
with  these  systems  is  frustratingly  awkward  compared  to 
on-site,  hands-on  operation  and  is  typically  10-20  times 
slower. 
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Major  contributions  to  the  design  of  mechanisms  and 
control  for  telerobotic  manipulators  originated  in  the  nu¬ 
clear  industry.  Examples  of  this  work  include  the  Argonne 
Mark-E4A  [1]  and  the  Brookhaven  Arm  [2].  This  early 
work  led  to  more  refined  bilateral  force-reflecting  manip¬ 
ulators  [3],  [4]. 

B.  Telepresence 

Telepresence  is  an  enhanced  form  of  teleoperation  that 
employs  an  immersive  and  transparent  user  interface,  per¬ 
mitting  the  user  to  work  with  high  effectiveness  in  inac¬ 
cessible  or  remote  environments.  In  telepresence  surgery, 
the  surgeon  works  at  a  telepresence  surgeon’s  workstation 
(TSW),  using  familiar  instruments  and  intuitively  respond¬ 
ing  to  the  stereoscopic  view,  proprioceptive  and  haptic  cues, 
and  sounds  that  are  provided  as  feedback  from  the  actual 
surgical  site.  With  telepresence,  the  user  can  remotely  per¬ 
form  complex  tasks  without  the  need  of  specialized  training. 

Using  modem  telemanipulator,  control,  and  imaging  ca¬ 
pabilities,  we  have  developed  systems  that  enable  the  full 
spectmm  of  surgical  tasks — such  as  cutting,  suturing,  and 
dissecting — normally  performed  by  surgeons.  Systems  with 
the  required  dexterity,  speed,  and  delicate  force  feedback 
have  not  been  previously  developed,  nor  has  a  human 
interface  methodology  for  making  their  use  natural  and 
effective. 

The  application  of  telepresence  principles  to  surgery  has 
several  potential  benefits: 

•  in  minimally  invasive  surgery  (MIS),  restoring  the 
hand-eye  coordination  that  is  lost  when  surgeons  use 
conventional  instruments,  thus  speeding  up  a  slow  and 
fatiguing  manual  process  and  bringing  the  benefits  of 
MIS  techniques  to  more  patients; 

•  in  microsurgery,  scaling  small  motions  and  forces 
to  the  optimal  range  of  human  perception,  thereby 
enabling  improved  performance  and  new  microsurgical 
procedures; 

•  bringing  lifesaving  surgical  care  to  isolated  patients  in 
rural  areas,  aboard  ship,  or  on  the  battlefield; 
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Fig.  1.  SRI  telepresence  surgery  principle. 

•  in  all  surgery,  opening  surgical  procedures  to  the 
benefits  of  telemedicine; 

•  in  training  of  medical  students,  interacting  with  com¬ 
puter  simulations  through  a  natural,  immersive  inter¬ 
face; 

•  in  preparation  for  surgery,  allowing  the  surgeon  to 
practice  surgery  on  a  “virtual  patient”  computer  model 
created  from  patient-specific  medical  image  data. 

C  Background 

Robots  have  begun  to  find  applications  in  surgery  in 
recent  years  [5],  [6].  For  example,  placement  of  probes 
in  the  brain  has  been  demonstrated  using  a  stereotactic 
robot  positioner  [7].  Typically,  the  robot  inserts  a  needle 
along  its  axis  deep  into  a  tumor.  The  “Robodoc”  orthopedic 
surgery  system,  developed  at  IBM  and  Integrated  Surgical 
Systems  [8],  produces  a  precise  cavity  in  the  femur  for  hip 
replacement.  Transurethral  resection  of  the  prostate  has  also 
been  performed  robotically  [9]. 

In  contrast  to  robot  automation,  earlier  researchers  have 
envisioned  the  potential  advantages  of  telemanipulation  in 
surgery.  These  advantages  include  a  concept  for  a  teleop¬ 
erator  surgery  system  in  which  the  surgeon  could  see  and 
feel  what  he  is  operating  on  [10].  Examples  of  prototype 
systems  are  a  handheld  master  with  a  pencil-like  tool  [11], 
a  finger-sized  manipulator  (four  links)  proposed  for  corneal 
transplant  surgery  [12],  and  a  three-axis  manipulator  for 
cutting  [13], 

D.  SRI  Telepresence  Principle  (Immersive  Interface) 

Alexander  [14]  described  a  concept  for  remote  surgery 
in  1973  and  made  a  conceptual  sketch  of  a  remotely 
operated  system  for  MIS.  The  sketch  shows  a  system  with 
master  hand  controllers  under  a  mirror  to  align  hand  and 
eye  axes  and  make  manipulation  more  intuitive.  The  first 
implementation  of  a  prototype  telepresence  surgery  system 
integrating  remote  vision,  sound,  hand  motion,  and  force 
feedback  was  developed  by  a  team  at  SRI  International  led 
by  P.  S.  Green  [15]. 


The  SRI  telepresence  principle  or  “Green  Telepresence 
System”  is  diagrammed  in  Fig.  1.  It  consists  of  two  mod¬ 
ules:  the  TSW  and  the  remote  surgical  unit  (RSU).  The 
TSW  contains  a  color  monitor  for  stereographic  viewing 
(the  operator  wears  passive  polarized  glasses  or  active 
shutter  glasses),  reflected  in  a  mirror  to  create  a  “virtual 
workspace.”  Beneath  the  mirror  are  mounted  stereo  speak¬ 
ers  and  (one)  hand-operated  master  manipulator,  which  is 
outfitted  with  a  surgical  instrument  handle. 

The  RSU,  shown  on  the  left  side  of  Fig.  1,  captures 
stereo  images  with  a  pair  of  color  video  cameras  converged 
at  about  8°,  the  same  interocular  viewing  disparity  that 
we  experience  with  a  visual  field  500  mm  in  front  of 
us.  Also  included  are  a  force-reflecting  slave  manipulator 
with  surgical  instrument  jaws  for  grasping  and  a  pair  of 
microphones  for  stereo  sound  pickup. 

The  master  and  slave  manipulators  are  placed  in  juxta¬ 
position  to  the  camera  and  to  the  display,  respectively,  so 
as  to  be  geometrically  identical.  Our  purpose  is  effectively 
to  transport  the  operator  to  a  remote  site  with  adequate 
fidelity  that  he  feels  that  he  is  actually  there.  This  results 
in  natural  and  instinctive  control  motions  by  the  operator, 
who  can  make  use  of  the  innate  and  instinctive  senses 
and  reactions  that  have  been  practiced  and  perfected  since 
birth. 

The  telepresence  surgeon  grasps  surgical  instrument  han¬ 
dles,  looks  down  into  a  three-dimensional  (3-D)  surgical 
field,  reaches  into  the  field  with  the  instruments,  and 
operates.  He  sees  the  tips  of  the  instruments  move  with 
his  hands  as  they  manipulate  the  tissues  and  feels  the 
tissues  resist.  To  the  surgeon,  this  looks  and  feels  like  the 
conventional  open  surgery  on  which  he  has  already  trained 
and  practiced.  At  the  slave  site  (i.e.,  the  patient  location), 
the  technique  may  be  MIS,  microsurgery,  or  open  surgery, 
and  the  patient  could  be  in  the  same  room  or  in  the  next 
state:  it  makes  no  difference.  Telepresence  provides  such 
a  compelling  sense  of  reality  that  the  surgeon  is  drawn 
immediately  into  the  work,  with  no  sensation  of  remote 
control.  Hand  motions  are  quick  and  precise. 
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Fig.  2.  Stereo  display  system.  Video  images  from  the  cameras 
are  acquired  and  buffered  at  30  frames/s  and  alternately  displayed 
at  120  Hz. 


Master  Slave 

Fig.  3.  Bilateral  control  system.  The  position  and  velocity  of 
each  joint  as  derived  from  the  joint  encoders  are  multiplied  by 
the  position  and  velocity  gains,  Kp  and  Kv,  and  input  to  both 
motors  to  provide  joint  torque. 


IL  Telepresence  Technology  Principles 

A.  Remote  Stereoscopic  Imaging 

Ideally,  visual  information  provided  to  the  surgeon  should 
take  full  advantage  of  the  resolution  capability  of  humans. 
Accomplishing  this  with  a  225  x  300-mm  (17-in)  video 
monitor  placed  500  mm  in  front  of  the  eye  would  require 
a  horizontal  resolution  of  2000-4000  lines  [16]. 

Emerging  high-definition  television  (HDTV)  systems  can 
now  provide  up  to  2000  lines  of  horizontal  resolution, 
but  HDTV  camera  components  are  not  yet  available  in 
configurations  suitable  for  telesurgery  applications.  For 
our  telepresence  systems,  we  have  selected  commercial 
equipment  with  the  highest  resolution  currently  available. 

The  stereo  imaging  subsystem  shown  in  Fig.  2  is  repre¬ 
sentative  of  those  used  in  our  telepresence  viewing  systems. 
Two  video  cameras  capture  the  stereoscopic  image  frames. 
Dual,  digital  frame  buffers  store  the  frame  pairs  at  30 
frames/s  and  alternately  provide  left-  and  right-frame  im¬ 
ages  to  the  operator’s  display  at  120  Hz.  Two  shuttering 
methods  are  commonly  used  to  ensure  that  each  eye  sees 
only  the  proper  (left  or  right)  image: 

•  a  synchronously  switched  polarizing  filter  in  close  con¬ 
tact  with  the  monitor  combined  with  passive,  polarized 
glasses  worn  by  the  observer; 

•  synchronously  switched  active  shutter  glasses  worn  by 
the  observer. 

Both  methods  pass  alternate  frames  to  the  left  and  right 
eye,  respectively.  The  60-Hz  presentation  at  each  eye 
provides  flicker-free  viewing. 

Alternative  configurations  use  separate  display  monitors 
for  each  eye  or  head-mounted  displays,  eliminating  the  need 
for  a  synchronous  shutter. 

Principal  design  factors  in  stereo  viewing  systems  are 
spatial  resolution,  field  of  view,  contrast  ratio,  convergence, 
depth  of  field,  and  (in  single-display  systems)  extinction 
ratio  for  the  stereo  shutter.  Head-aimed  systems  intro¬ 
duce  slew  rate  limitations  for  pitch  and  yaw  as  well  as 
convergence.  Distortions  in  viewing  geometry,  which  lead 
to  misperceptions,  can  be  introduced  at  both  the  camera 
and  display  [17].  Supplementary  visual  information  as  a 
heads-up  display,  patient  monitor,  or  image  overlays  can 
be  introduced  to  augment  the  visual  information.  Well- 
designed  viewing  systems  permit  the  surgeon  to  position 
tools  accurately  at  the  desired  location  and  create  a  sense 
of  “immersion.” 


B.  Force-Reflecting  Master-Slave  Manipulators 

For  efficient  performance  in  surgery,  the  handles  at 
the  master  manipulators  should  act  and  feel  like  familiar 
surgical  tools.  In  this  way,  prior  surgical  training  is  rapidly 
transferred  to  the  telerobotic  system.  The  master  manip¬ 
ulator  should  impose  a  low  mass  load  (25-35  g)  on  the 
operator’s  hand  and  have  an  adequate  range  of  motion  in  six 
degrees  of  freedom  (dof)  [18].  It  should  have  unnoticeable 
friction  and  damping,  move  freely  at  up  to  1  m/s,  and  have 
a  bandwidth  of  at  least  10  Hz  [19],  [20]  to  avoid  tiring  the 
operator.  Higher  bandwidth  is  desirable  for  conveying  the 
feel  of  vibration  and  hard  contact. 

Master-slave  manipulator  design  differs  from  industrial 
robot  design  in  that  the  mechanism  must  have  low  inertia. 
To  accomplish  this,  the  actuators  must  be  located  appropri¬ 
ately  so  that  their  mass  does  not  cause  undesirable  loading. 
Friction  is  minimized  by  using  few  stages  of  gearing, 
iow-friction/low-inertia  actuators,  and  low-friction  bearings 
throughout. 

Force  feedback  is  provided  on  all  axes  of  the  manipulator, 
including  the  gripper  closing  force.  Forces  and  torques 
generated  as  the  tool  touches  objects  in  the  course  of 
its  work  are  reproduced  at  the  operator’s  hand  by  the 
control  system.  Soft  objects  must  feel  soft,  and  hard  ones 
hard.  The  force  feedback  from  the  actions  of  probing, 
dissecting,  cutting,  and  suturing  should  be  clear  and  natural 
to  maximize  surgical  performance. 

Principal  design  factors  in  manipulators  are  degrees  of 
freedom  and  working  volume.  Other  design  specifications 
such  as  accuracy,  backlash,  friction,  inertia,  and  bandwidth 
vary  with  position  in  working  volume. 

C  Control  Systems 

Bilateral  control  of  manipulators  encompasses  research  in 
appropriate  control  objectives,  in  algorithmic  implementa¬ 
tions,  and  in  stability  and  robustness  of  controllers.  Bilateral 
control  objectives  and  implementations  have  taken  various 
forms.  A  well-known  bilateral  control  objective  designed 
to  achieve  some  force  reflection  is  that  of  bilateral  position 
tracking  [21],  as  shown  in  Fig.  3.  This  technique  is  easily 
implemented  and  can  be  designed  to  be  robust,  but  the  user 
is  burdened  with  the  total  inertia  of  both  the  master  and 
slave  on  his  hand. 

Another  common  goal  for  bilateral  control  of  manipula¬ 
tors  is  ideal  (or  transparent)  coupling,  where  the  objective  is 
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to  provide  perfect  position  tracking  between  the  master-  and 
slave-end  effectors  while  providing  perfect  reflection  of  the 
force  at  the  slave-end  effector  to  the  master-end  effector. 
Several  control  algorithms  have  been  developed  to  approx¬ 
imate  this  ideal  coupling.  An  early  algorithm  designed 
to  achieve  ideal  coupling  is  the  position-forward/force- 
back  control  scheme  [2],  where  the  slave  manipulator  is 
controlled  to  track  the  master’s  position  while  the  master 
manipulator  is  controlled  to  track  the  slave’s  force.  A 
modification  to  position  tracking  that  includes  bidirectional 
force  measurements  [22]  was  difficult  to  tune  to  ensure 
stability.  Introducing  planned  compliance  into  the  controller 
allowed  for  higher  force  reflection  gains  while  retaining 
stability  [23].  More  recently,  a  physically  motivated  control 
algorithm  was  developed  for  realizing  ideal  kinesthetic 
coupling  in  a  single-degree-of-freedom  system  based  on  po¬ 
sition,  velocity,  acceleration,  and  force  measurements  [24]. 

A  generalization  of  the  control  objective  is  that  of  me¬ 
chanical  impedance  modification.  Here,  the  controller  is 
designed  to  modify  the  impedance  of  the  environment  as 
perceived  by  the  operator  [25]-[28].  Mechanical  impedance 
modification  is  at  the  heart  of  bilateral  control  systems  that 
act  as  “extenders”  to  increase  human  strength  [29]  and 
micromanipulators  that  scale  down  motion  and  scale  up 
force  [30]. 

Stability  robustness  of  bilateral  controllers  when  coupled 
to  the  operator  and  to  varying  environments  is  of  critical 
importance  [31],  [32].  The  requirement  for  stability  of  the 
closed-loop  control  system  when  coupled  to  passive,  but 
otherwise  arbitrary,  loads  and  when  power  scaling  is  not 
required  is  known  to  be  a  passivity  constraint  on  the  control 
system  [33]-[35].  When  the  bilateral  controller  is  intended 
to  scale  power,  the  condition  for  robustness  when  the 
manipulator  is  coupled  to  an  arbitrary,  passive  environment 
has  been  derived  in  terms  of  structured  singular  values  of 
a  scattering  representation  of  the  system  [36]. 

Principal  design  factors  in  control  systems  are  band¬ 
width/response  time,  controller  order,  update  rate,  and 
hardware  implementation.  Desirable  bonuses  are  reductions 
of  parasitic  mass,  friction,  and  damping  forces  felt  by  a 
human  hand  on  the  tool  through  the  incorporation  of  force 
sensors.  Using  the  ideal  kinesthetic  coupling  algorithm  [25], 
we  have  demonstrated  a  50%  reduction  in  hand-tool  mass. 

D.  Aural  Feedback  System 

Stereo  sound  feedback  can  enhance  immersion  and  per¬ 
formance  in  remote  tasks  by  allowing  the  surgeon  to  1) 
assess  his  actions  via  the  sounds  they  create,  2)  take 
advantage  of  audible  spatial  location  cues,  and  3)  talk  to 
a  remote  assistant  or  a  team  in  the  procedures  as  if  he  were 
actually  there. 

Our  remote  open- surgery  RSU’s  incorporate  two  micro¬ 
phones  positioned  above  and  to  the  sides  of  the  work 
site  itself.  Two  speakers  are  located  in  the  TSW  just 
above  and  to  the  sides  of  the  hand-controller  region — the 
apparent  work  site.  Alternatively,  a  pair  of  earphones  can 
be  used.  Either  arrangement  provides  stereophonic  sounds 
that  seem  to  emanate  from  the  work  site  and  that  are 


generally  associated  with  the  side  of  the  work  site  from 
which  they  originated.  The  distance  between  the  left  and 
right  microphone  elements  is  approximately  the  same  as 
the  distance  between  the  speakers  that  reproduce  the  sound. 
Design  issues  are  the  sensitivity,  frequency  response,  and 
location  of  the  microphones  and  speakers. 

E.  Telecommunication  Links  for  Telepresence 

The  communication  link  between  the  TSW  and  RSU  can 
take  a  variety  of  forms,  depending  on  system  requirements, 
ranging  from  copper  cabling  to  microwave  radio  links 
and  optical  fiber  connections.  In  each  case,  however,  it 
is  important  to  consider  the  separate  requirements  for  the 
servo  and  video  communication  channels.  The  servo  data 
are  inherently  digital  and  require  a  bandwidth  of  only  about 
300  kb/s  but  must  have  a  latency  of  less  than  about  5  ms. 
(Time  delays  produce  a  viscous  drag  phenomenon  in  the 
servo  system,  which  can  be  compensated  for  by  using  phase 
lead  up  to  about  5  ms.) 

Our  camera  outputs  are  currently  analog,  so  the  video 
channel  can  be  analog  or  digital,  and  it  can  also  tolerate 
greater  latency  (up  to  about  50  ms)  before  annoying  de¬ 
lays  become  noticeable  in  updating  the  stereo  image.  To 
implement  a  fully  digital  communication  link,  we  trans¬ 
mit  the  stereo  video  data  by  using  conventional  video 
codecs,  which  require  one  DS3  channel  (45  Mb/s)  for 
each  video  stream.  As  a  result  of  our  latency  require¬ 
ment,  Motion  Pictures  Experts  Group  and  other  multiframe, 
high-compression  schemes  are  not  acceptable  in  the  video 
communication  channel.  For  local  operation  within  the 
same  hospital  facility,  analog  video  transmission  over  coax¬ 
ial  cabling  may  be  an  appropriate  option.  Digital  video 
transmission  is  clearly  required  for  transmission  over  wider 
networks. 

Design  parameters  of  the  communication  link  are  time 
delay,  image  distortion  (through  compression  algorithms), 
and  signal  bandwidth  for  manipulator  position  and  force 
signals. 

F.  Evaluation  of  Human  Factors 

Pioneering  human  performance  work  was  conducted  by 
Fitts  [37],  who  showed  that  the  time  for  manual  positioning 
tasks  is  proportional  to  the  logarithm  of  the  ratio  of  the 
distance  moved  divided  by  the  target  width.  He  called  this 
the  task  difficulty  and  measured  it  in  bits.  As  target  width 
decreases,  task  time  increases,  and  the  slope  of  this  line 
gives  the  time  constant  of  the  exponentially  decreasing 
hand-eye  settling  error.  Fitts’  law  has  been  shown  to  apply 
to  tasks  with  telemanipulators  having  two  to  six  dof  [38] 
and  to  force-reflecting  telemanipulators  by  breaking  Fitts’ 
positioning  task  into  an  open-loop  move  and  a  vision/feel 
guided  subtask  [39],  [40].  Complex  telemanipulator  tasks 
can  be  further  broken  down  into  elemental  subtasks  (e.g., 
move,  turn,  position,  grip,  fit)  to  compare  force,  vision, 
or  other  experimental  conditions  better  [41].  This  permits 
distinguishing  open-loop  tasks  (move  and  turn  elements), 
which  do  not  depend  upon  force  sensing,  from  closed-loop 
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Jasks  (position  and  fit  elements),  which  depend  on  force  and 
3-D  vision  for  completion.  Depth-matching  accuracy  was 
measured  (the  Verhoff  test)  to  compare  3-D  displays  with 
different  interocular  viewing  distances  [42].  Sheridan  [43] 
summarizes  contributions  to  manual  performance  measure¬ 
ment  with  different  hand  controls,  tasks,  control  systems, 
viewing  conditions,  and  time  delays. 

III.  Current  Application  Areas 

A.  Telepresence  Minimally  Invasive  Surgery  System 

1 )  Background:  In  recent  years,  MIS  has  largely  replaced 
conventional  open  surgical  techniques  for  removal  of  the 
gallbladder  and  the  appendix  and  for  other  relatively  simple 
abdominal  procedures.  MIS  methods  are  slow  to  be  adapted 
to  more  complex  procedures,  however,  because  current  la¬ 
paroscopic  instruments  are  very  awkward  and  cumbersome 
to  use  compared  to  those  of  open  surgery. 

A  primary  limitation  is  that  the  instruments  are  operated 
through  an  incision  in  the  abdominal  wall,  which  serves  as  a 
fulcrum,  with  the  surgeon’s  hand  on  the  opposite  side  of  the 
fulcrum,  away  from  the  surgical  site.  Moreover,  the  video 
axis  is  not  aligned  with  the  hand  axis,  and  the  surgeon 
watches  the  instrument  tips  on  a  video  screen  across  the 
operating  room  table,  an  arrangement  that  makes  hand-eye 
coordination  very  difficult  and  engenders  a  feeling  of  dis¬ 
connectedness.  Consequently,  every  motion  is  deliberate 
rather  than  intuitive,  and  relatively  simple  maneuvers,  such 
as  suturing,  are  time  consuming  and  frustrating. 

Telepresence  technology  has  the  potential  to  overcome 
these  limitations,  enabling  the  expanded  application  of  MIS 
to  a  wider  range  of  more  complex  procedures  and  thus 
bringing  the  benefits  of  MIS  to  many  more  patients. 

2)  Work  at  Other  Laboratories:  Positioning  the  endo¬ 
scope  during  laparoscopy  requires  an  additional  assistant 
during  surgical  procedures.  To  replace  this  function  as  well 
as  to  provide  better  control  for  the  surgeon,  several  robotic 
scope  positioners  have  been  developed  [44]-[47].  Control 
variations  include  speech  recognition,  head  motion,  and 
hand  and  foot  switches. 

An  MIS  manipulator  system  with  remote-center  ma¬ 
nipulators  has  been  demonstrated  in  Germany  [48].  This 
approach  includes  force-reflecting  manipulators,  but  the 
video  display  system  does  not  maintain  a  natural  hand-eye 
axis. 

3}  Work  at  SRI:  At  SRI,  we  have  developed  a  prototype 
telepresence  (T)MIS  system  designed  to  overcome  the 
loss  of  hand-eye  coordination  and  touch  sensitivity  that 
occurs  with  conventional  MIS  technology.  In  designing  the 
prototype,  we  paid  careful  attention  to  the  human  factors 
involved  in  surgery  and  to  the  special  demands  of  MIS. 

The  TSW  shown  in  Fig.  4  employs  a  mirror  to  create 
a  “virtual  workspace”  beneath  the  hand-operated  master 
controllers. 

The  TSW  has  the  following  characteristics: 

•  master  manipulators:  each  with  a  polar  coordinate 
system  with  four  dof  (roll,  pitch,  yaw,  and  extension) 
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Fig.  4.  Telepresence  surgeon  workstation  for  the  MIS,  micro¬ 
surgery,  and  open-surgery  systems. 

with  tool  activation;  workspace  200  nun  high  and  wide 
by  100  mm  in  extension;  continuous  force  of  7  N; 
friction  approximately  0.3  N; 

•  handgrips:  loop  handles  like  those  found  on  surgical 
forceps; 

•  visual  display:  17-in  Tektronix  color  stereo  display 
(120  Hz)  with  integrated  synchronous  shutter  and 
polarized  glasses; 

•  audio  system:  stereo  speakers  providing  aural  feedback 
from  the  remote  work  site. 

The  RSU  of  the  TMIS  system  (Fig.  5)  employs  novel 
force-reflecting  slave  manipulators  that  maintain  a  remote 
center  of  rotation  aligned  with  the  tool  entry  point  (i.e.,  the 
surgical  incision)  in  the  patient’s  abdominal  wall. 

The  RSU  and  TSW  each  incorporates  a  pair  of  manip¬ 
ulators,  each  with  the  same  four  dof  (plus  tool  actuation) 
used  in  conventional  MIS  instruments.  With  telepresence, 
the  surgeon’s  hand  is  effectively  on  the  correct  side  of  the 
fulcrum,  so  the  tool  tip  precisely  follows  the  surgeon’s 
hand  motion  in  a  totally  intuitive  manner.  The  bilateral 
control  system  provides  for  magnified  motion  and  sensitive 
force  reflection.  The  low-inertia  electromechanical  design 
provides  a  high-bandwidth,  low-drag  system  with  surprising 
realism.  System  operation  is  intuitive  and  natural,  no  oper¬ 
ator  training  is  required,  and  remote  tasks  can  be  performed 
with  a  high  degree  of  dexterity  and  skill. 

The  characteristics  of  the  TMIS  RSU  are  as  follows: 

•  slave  manipulators:  each  with  a  polar  coordinate  sys¬ 
tem  with  four  dof  (roll,  pitch,  yaw,  and  extension)  plus 
tool  actuation;  workspace  250  mm  in  pitch  and  yaw, 
150  mm  in  extension;  continuous  force  of  7  N; 

•  tools:  interchangeable  needle  holders,  forceps, 
graspers,  scissors  (Fig.  5)  to  enable  a  range  of  MIS 
procedures; 
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(b) 

Fig.  5.  (a)  Telepresence  minimally  invasive  surger>'  system  and 
remote  surgical  unit,  (b)  Set  of  replaceable  tools. 


grasping  and  cutting  instruments  in  a  pencil-like  grip, 
with  their  palms  supported,  to  optimize  fine  motor  control 
and  minimize  hand  tremor  and  fatigue.  One  instrument 
generally  holds  or  applies  tension  while  the  other  cuts  or 
applies  a  needle. 

Microsurgical  tasks  and  skills  used  in  surgical  proce¬ 
dures  across  numerous  specialties  include  ophthalmology, 
otology,  digit-reattachment  surgery,  microvascular  surgery, 
urology,  obstetrics  and  gynecology,  neurosurgery,  and  min¬ 
imally  invasive  surgery  [50],  [51].  The  tasks  and  skills 
required  in  all  these  specialties — specifically  microposition¬ 
ing,  making  incisions,  microdissection,  and  suturing  small 
vessels — are  common,  indicating  the  wide  applicability  of 
a  microtelemanipulation  system  developed  for  this  purpose. 

Human  limitations  include: 

1)  procedures  that  are  inherently  physically  exhausting, 
mainly  due  to  the  long  hours  of  concentrated  work 
in  looking  through  microscope  eyepieces,  often  with 
the  surgeon  sitting  in  an  awkward  position; 

2)  four-dof  tool  motion  due  to  the  palm-on-table  posture 
that  limits  tool  orientation; 

3)  hand  tremor  at  high  image  magnification; 

4)  no  feel  at  all  (no  force  feedback)  at  high  magnifica¬ 
tion. 


•  control  system:  bilateral  position-position  on  rotation 
and  extension  axes,  combined  bilateral  position  and 
force  back  (active  force  sensing)  on  pitch  and  yaw; 
motion  and  force  scaling; 

•  imaging:  wolf  stereo  endoscope  with  xenon  arc  source, 
magnification  variable  from  5x  to  30x. 

Preliminary  animal  experiments  and  in  vitro  studies  have 
been  completed  using  the  prototype  TMIS  system.  The 
animal  experiments  were  carried  out  using  a  porcine  model, 
and  cholecystectomies  and  other  abdominal  procedures 
were  successfully  performed  with  the  system,  with  no 
aborted  attempts  or  complications.  Participating  surgeons 
confirmed  that — even  with  the  four-dof  prototype  TMIS 
system — ^the  procedures  had  a  more  “intuitive  feel”  than 
with  conventional  MIS  instruments. 

To  more  objectively  evaluate  the  performance  of  the 
TMIS  prototype,  we  have  carried  out  a  set  of  controlled  in 
vitro  experiments  in  which  surgical  residents  performed  a 
series  of  anastomosis  and  knot-tying  tasks  by  using  both  the 
TMIS  system  and  conventional  MIS  tools,  in  randomized 
order  [49].  In  both  the  anastomosis  task  (suturing  together 
two  pieces  of  synthetic  blood  vessel)  and  the  knot-tying 
task,  performance  with  the  TMIS  system  was  approximately 
30%  faster  than  with  conventional  MIS  instruments.  In 
addition,  the  leakage  rate  from  the  completed  anastomoses 
was  about  25%  less  with  the  TMIS  system. 

B,  Telepresence  Microsurgery  System 

I)  Background:  In  current  practice,  microsurgery  in¬ 
volves  dexterous  manipulations  on  small  tissues  viewed 
through  a  stereo  microscope.  Surgeons  hold  special 


Telepresence-based  microsurgery  has  the  potential  to 
provide  the  surgeon  with  a  magnified  workspace  in  which 
he  can  comfortably  work  with  his  hands  on  full-sized  instru¬ 
ment  handles,  using  normal  hand  motions  and  experiencing 
the  feel  he  would  expect  from  the  magnified  tissues  that  he 
sees. 

2 )  Research  at  Other  Institutions:  Several  teams  are 
developing  microtelemanipulation  systems  for  ophthalmic 
surgery  applications.  Ophthalmic  systems  are  characterized 
by  a  single  manipulator  arm  with  a  needle-like  probe 
without  force  feedback  and  a  view  of  the  workspace 
through  a  surgical  stereo  microscope.  Applications  include 
radial  keratotomy  [52],  [53]  and  insertion  of  a  hypodermic 
probe  into  retinal  vessels  to  measure  pressure  and  to 
inject  drugs  [54],  [55].  Systems  with  a  position-controlled 
master  and  amplified  force  feedback  have  been  developed 
for  vitreo-retinal  (inner  eye)  surgery  [56]  and  general 
eye  surgery  [57].  Force-amplifying  micromanipulator 
systems  employing  video  viewing,  and  hence  applicable  to 
remote  operation,  have  demonstrated  cutting  a  simulated, 
polymer  eye  [58]  and  sawing  a  4-mm-diameter  soft 
tube  [59]. 

To  date,  none  of  the  microtelemanipulation  research  has 
addressed  the  two-handed  requirements  of  1)  microvascular 
suturing,  where  a  needle  holder  is  held  in  one  hand  and  a 
pickup  (forceps  or  tweezers)  in  the  other,  or  2)  dissection, 
where  a  cutting  tool  is  held  in  one  hand  and  tension  applied 
with  the  other  with  a  pickup. 

3}  Work  at  SRI:  To  address  the  needs  for  performing 
scaled-down  procedures,  the  TSW  (with  the  TMIS  sys¬ 
tem)  has  been  combined  with  the  micromanipulator  RSU 
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Fig.  6.  Microsurgery  remote  surgical  unit. 


shown  in  Fig.  6.  The  micromanipulator  RSU  includes  the 
following  features: 

•  slave  micromanipulators:  a  pair  of  precision  manipula¬ 
tors  without  gearing  on  translational  axes;  a  workspace 
of  100  X  100  X  100  mm;  continuous  force  of  2  N,  four 
dof  plus  gripper,  and  a  resolution  of  7  /xm; 

•  tools:  microneedle-holder  and  micropickup; 

•  control  system:  bilateral,  reduces  motion  scale  and 
increases  force  scale,  incorporates  forces  measured 
with  triaxial  load  cells; 

•  imaging:  stereo  zoom  microscope  with  three  charge- 
coupled  device  (CCD)  cameras  mounted  in  eyepieces, 
ring  light  source,  magnification  variable  from  6x  to 
60x. 

The  prototype  microsurgery  system  has  been  tested  with 
ex  vivo  tasks  similar  to  those  required  for  surgical  proce¬ 
dures  such  as  cutting,  grasping,  suturing,  and  knot  tying. 
Initial  animal  testing  has  been  done  on  a  rat  model  in 
which  end-to-end  anastomosis  of  the  femoral  artery  (ap¬ 
proximately  1  mm  in  diameter)  has  been  completed  with 
ten  rats.  Full  patency  was  obtained  in  all  cases. 


C  Open-Surgery  Demonstration  System 

The  driving  force  behind  military  interest  in  remote 
surgery  is  that  90%  of  all  deaths  in  wartime  occur  on 
the  battlefield,  before  surgical  care  can  reach  the  casualty. 
Rapid  evacuation  has  been  beneficial  in  decreasing  the 
number  of  casualties  dying  of  wounds  after  reaching  a  field 
hospital,  but  it  has  not  significantly  reduced  the  killed-in- 
action  rate.  The  most  likely  reason  that  battlefield  mortality 
remains  high  is  that  definitive  surgical  care  is  not  available 
during  the  critical  first  hour  after  wounding,  appropriately 
termed  the  “golden  hour”  by  trauma  surgeons. 

Nonbattlefield  applications  of  remote,  open-surgery  sys¬ 
tems  include  the  following: 

•  military  applications  (ship  to  ship,  space  station); 

•  civilian  applications  (rural  intervention,  remote  con¬ 
sultation). 


(b) 


Fig.  7.  (a)  Demonstration  system  remote  surgical  unit,  (b)  Set  of 
replaceable  tools. 


Of  course,  the  principles  of  the  “open-surgery”  system 
are  also  applicable  to  nonmedical  tasks  such  as  satellite 
servicing,  underwater  science,  and  ocean  engineering. 

The  SRI  telepresence  demonstration  system  [60]  employs 
the  same  TSW  unit  as  that  described  here  for  the  TMIS 
system  (see  Fig.  4).  The  demonstration  system  RSU,  shown 
in  Fig.  7,  includes  two  force-reflecting  slave  manipulators 
identical  to  those  of  the  TSW  except  with  surgical  instru¬ 
ment  tips  instead  of  handles. 

The  open-surgery  demonstration  system  RSU  includes 
the  following  features: 

•  manipulators:  each  with  a  polar  coordinate  system 
with  four  dof  (roll,  pitch,  yaw,  and  extension)  with 
tool  actuation;  workspace  200  mm  high  and  wide  by 
100  mm  in  extension;  continuous  force  of  7  N;  friction 
of  approximately  0.3  N; 

•  tools:  interchangeable  needle  holders,  forceps  (several 
types),  graspers,  and  scissors,  adapted  from  actual 
surgical  tools,  to  enable  a  range  of  procedures  to  be 
performed; 

•  control  system:  bilateral  control  to  scale  motions  and 
forces  and  incorporate  forces  measured  with  strain 
gauges; 

•  imaging:  triple-CCD  remote-head  cameras  with  500- 
mm  standoff;  manually  operated  zoom  lenses  and 
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incandescent  lamps,  magnification  variable  from  half¬ 
size  (0.5 x)  to  3x. 

Using  the  SRI  demonstration  system,  we  have  success¬ 
fully  performed  a  series  of  remote  surgical  procedures  on 
pigs.  These  first  ever  remote  surgeries  included  treatment 
of  a  variety  of  simulated  abdominal  and  vascular  injuries 
and  were  performed  by  surgeons  experienced  in  trauma 
management.  The  successful  results  of  these  experiments 
clearly  demonstrate  the  feasibility  of  telepresence  surgery 
for  emergency  stabilization  of  battlefield  casualties.  In  ad¬ 
dition,  project  engineers  closely  observed  the  experimental 
procedures  and  held  in-depth  discussions  with  the  surgeons 
to  gain  important  feedback  and  suggestions  concerning 
system  performance  and  features.  The  results  of  these 
experiments  and  discussions  have  served  as  an  important 
guide  in  the  development  of  the  advanced  surgery  system 
(MEDFAST). 

Surgeries  were  performed  on  four-  to  six-month-old 
female  Hampshire  farm  pigs  weighing  60-80  pounds.  All 
procedures  attempted  were  completed  successfully,  includ¬ 
ing  cholecystectomy,  nephrectomy,  repair  of  gastrotomy 
(simulated  stomach  injury),  control  of  liver  laceration, 
repair  of  femoral  artery  laceration,  aortic  interposition  graft, 
and  repair  of  bladder  laceration. 

The  quality  of  the  surgical  procedures  performed  was 
nearly  identical  to  that  of  those  performed  by  conventional 
hands-on  procedures,  and  task  times  were  only  2.5  x  those 
of  hands-on  procedures  [61].  Target  acquisition  measured 
using  Fitts’  law  was  about  50%  slower  than  hands-on 
but  about  twice  as  fast  as  nonimmersive  (teleoperation) 
interfaces  [62]. 

D.  MEDFAST  Advanced  Open-Surgery  System 

1 )  Advanced  Telepresence  Surgery  System:  We  have  de¬ 
veloped  an  advanced  telepresence  surgery  system  with  three 
components: 

i)  a  medic-positionable  RSU  to  be  installed  in  an 
armored  ambulance  (the  MEDFAST  vehicle)  with 
a  pair  of  six-dof  manipulators; 

ii)  a  TSW  incorporating  a  pair  of  six-dof  force- 
reflecting  hand  controllers,  stereo  video  display, 
patient  monitor  display,  and  other  display  and 
control  devices; 

iii)  a  low-latency,  two-way,  digital  communication  link 
between  the  RSU  and  the  TSW. 

The  link  includes  video  digitization,  compression,  encod¬ 
ing  of  all  signals  (video,  servo  commands,  audio,  patient 
monitoring  information),  and  formatting  of  the  data  stream 
for  transmission  via  microwave  or  fiber-optic  links.  The 
TSW  and  RSU  of  the  MEDFAST  system  are  illustrated  in 
Figs.  8  and  9. 

Major  components  of  the  MEDFAST  system  are: 

•  manipulators:  two  master  and  two  slave,  each  with 
six  dof,  using  a  three-axis  wrist  and  single-axis  el¬ 
bow;  push-pull  tool  actuation  for  the  end  effector; 
continuous  force  of  13.5  N; 


Fig.  8.  MEDFAST  telepresence  surgeon’s  workstation. 

•  workspace:  400  x  400  x  400  mm,  with  rotational 
motion  of  360""  on  tool  roll  and  yaw  axis,  150°  pitch 
axis; 

•  handgrips:  pistol-grip  handles  with  a  pair  of  finger 
loops  that  operate  proportionally; 

•  tools:  manually  or  automatically  changeable  tools 
(scalpel,  pickups,  hemostat,  needle  holder  with  200-N 
grip,  and  scissors); 

•  control  system:  bilateral,  reduces  forces  on  master  to 
50%  of  slave  force,  provision  to  incorporate  forces 
measured  with  a  triaxial  load  cell  or  six-axis  load  cell 
with  through-hole  for  tool  actuation; 

•  imaging:  triple-CCD  remote-head  cameras  with  re¬ 
motely  operated  zoom  lenses  and  tungsten-halogen 
lamps,  magnification  variable  from  half-size  (0.5  x )  to 
5  X ,  remote  focus  and  brightness  controls; 

•  visual  display:  120-Hz  stereo  display  with  five- 
segment  polarizer  and  passive  polarized  glasses; 

•  panoramic  display:  an  additional  (low-bandwidth) 
video  camera  and  heads-up  display  to  provide  a 
panoramic  view  of  the  medic  and  patient  inside  the 
MEDFAST  vehicle. 

2}  Communication  Link:  We  have  developed  a  two-way 
communication  link  that  provides  a  wireless  or  network 
connection  between  the  TSW  and  the  RSU  in  the  MED¬ 
FAST  vehicle.  The  RSU  communication  link  shown  in 
Fig.  10  includes  subsystems  for  digital  data  compression, 
multiplexing,  and  microwave  or  network  transmission.  The 
TSW  link  performs  the  inverse  operations. 

a )  Video  and  audio  subsystems:  The  RSU  includes 
video  cameras  for  the  surgeon’s  stereo  and  panoramic 
views,  and  their  output  video  streams  must  be  digitized  (via 
codecs)  prior  to  radio  transmission.  Because  the  latency  and 
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Fig.  9.  (a)  MEDFAST  remote  surgical  unit,  (b)  Replaceable 
needle  holder  and  forceps. 


resolution  of  the  surgeon’s  3-D  view  are  critical  to  system 
performance,  we  undertook  a  careful  review  of  available 
video  codecs.  Our  laboratory  testing  of  the  codecs  included 
comparative  and  absolute  resolution  measurements  using 
calibrated  test  charts  and  latency  tests  using  switched  diode 
light  sources  and  digital  timing  equipment.  We  evaluated 
several  high-performance  codec  models  and  found  that  the 
Alcatel  model  1745  VC  provided  the  best  performance  in 
terms  of  our  low-latency,  high-resolution  requirements,  in 
part  because  it  performs  minimal  signal  compression.  We 
therefore  selected  the  1745  VC  codec  for  processing  the 
3-D  video.  The  1745  VC  includes  a  pair  of  high-fidelity 
audio  subchannels  that  provide  stereo  sound  feedback  to 
the  surgeon. 

Latency  is  less  of  aii  issue  for  the  surgeon’s  panoramic 
view,  and  we  have  chosen  to  compress  the  panoramic 
video  (via  a  PictureTel  PCS- 100  unit)  to  fractional  DSl 
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Servo 
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Fig.  10.  RSU  communication  link. 


bandwidth.  The  PCS-100  units  provide  duplex  transmission 
with  good  image  and  sound  quality. 

b)  Servo  communication  subsystem:  Servo  control  data 
are  passed  between  the  TSW  and  RSU  servo  controllers 
via  a  DSl  channel.  Full  duplex  position  data  streams,  in 
combination  with  force  sensor  data,  ^e  sent.  The  hard¬ 
ware  consists  of  commercial  servo  controller  cards  for  16 
axes  and  a  T1  serial  port  card  in  Pentium-based  personal 
computer  (PC)  hosts  at  the  RSU  and  TSW.  To  implement 
the  servo  communication  link,  we  have  developed  real¬ 
time  interface  software  for  the  host  PC  that  transfers 
data  between  the  hosts  via  the  T1  ports  and  continuously 
monitors  data  integrity. 

c)  Radio  subsystem:  The  radio  connection  between 
the  TSW  and  the  RSU  is  implemented  with  digital 
microwave  radio  units  (Alcatel  model  MDR-4208e).  This 
radio  has  a  transmission  capacity  of  two  DS3  channels  and 
two  DSl  side  channels. 

We  demonstrated  operation  of  the  MEDFAST  surgery 
system  with  the  radio  link  between  the  MEDFAST  vehicle 
and  base  station  on  top  of  a  nearby  SRI  building.  Operation 
of  the  radio  link  was  also  demonstrated  over  1,8  km  at  SRI’s 
radio  field  test  site  in  Los  Banos,  CA,  in  December  1995. 

d)  Network  subsystem:  In  August  1996,  operation  of 
the  existing  four-dof  SRI  telesurgery  system  was  demon¬ 
strated  with  the  TSW  located  at  Lawrence  Livermore 
National  Laboratory  (Livermore,  CA)  and  the  RSU  lo¬ 
cated  at  SRI  (Menlo  Park,  CA).  The  TSW  and  RSU 
were  connected  via  our  codecs  and  a  multiple-wavelength 
all-optical  fiber  network  (including  optical  switches  and 
amplifiers  spanning  300  km)  that  was  provided  by  the 
National  Transparent  Optical  Network  Consortium  project 
at  the  national  laboratory,  funded  by  the  Defense  Advanced 
Research  Project  Agency.  An  additional  time  delay  imposed 
by  the  network  implementation,  approximately  1-2  ms,  was 
compensated  for  by  increasing  the  lead  factor  in  the  servo 
algorithm.  Demonstration  participants  were  able  easily  and 
successfully  to  carry  out  remote  tasks,  unimpeded  by  the 
300-km  distance. 

E.  Testbed  for  Open  Telepresence  Surgery  at  the  Uniformed 
Services  University  of  the  Health  Sciences  (USUHS) 

The  objective  of  this  development  work  is  to  provide 
the  Department  of  Defense  (DOD)  with  a  fully  functional, 


H!LL  AND  JENSEN:  TELEPRESENCE  TECHNOLOGY  IN  MEDICINE 


577 


prototype  telepresence  surgical  unit  for  use  at  USUHS  in 
Bethesda,  MD.  This  system  will  allow  key  U.S.  Army, 
Navy,  and  Air  Force  surgeons  from  all  surgical  specialties 
to  evaluate  telepresence  surgery  and  develop  doctrine  for 
its  use.  Further,  the  system  can  be  used  to  explore  methods 
such  as  surgical  simulation  for  enhancing  the  core  curricula 
of  undergraduate  and  graduate  medical  education. 

To  accomplish  these  objectives,  SRI  has  installed  a  six- 
dof  MEDFAST  telepresence  surgery  system  at  USUHS 
and  is  providing  system  maintenance  and  repair  during 
the  contract  period.  In  cooperation  with  USUHS  medical 
researchers,  SRI  is  devising  and  conducting  scientific  tests 
to  evaluate  military-specific  surgical  applications  of  the 
system. 

The  objectives  for  the  USUHS  testbed  project  will  be  to: 

•  develop  appropriate  measures  of  surgical  task  perfor¬ 
mance; 

•  develop  mission  needs  evaluations  for  DOD  applica¬ 
tions  of  telepresence  surgery; 

•  evaluate  the  MEDFAST  six-dof  system  and  modify 
designs  as  appropriate  for  USUHS  applications; 

•  initiate  scientific  studies  of  telesurgery  at  USUHS. 

F.  Surgical  Simulation 

1 )  Background:  One  of  the  significant  challenges  facing 
medicine  is  training  for  surgical  procedures  [63].  These 
tasks  require  cognitive,  perceptual,  manual  dexterity,  and 
hand-eye  coordination  skills.  Current  training  practices 
involve  hands-on  training  on  animals  and  human  cadavers, 
both  of  which  are  becoming  more  expensive  and  increas¬ 
ingly  difficult  to  obtain. 

Major  simulation  components  include  generation  of  the 
stereo  visual  image  [64],  force  reflection  on  the  handheld 
tools  (haptic  feedback),  and  synthesized  sounds.  Required 
anatomical  modeling  components  include  contact  detection 
of  tool  and  tissue,  deformation  of  compliant  tissue  to  force, 
and  change  in  shape  due  to  tearing,  cutting,  and  puncture. 

2)  Research  at  Other  Institutions:  Principal  work  in  this 
area  has  been  to  demonstrate  environments  for  skill  transfer 
using  simulations  based  on  the  workbench  approach.  Little 
work  has  been  done  on  training  effectiveness,  comparison 
of  simulator  training  with  hands-on  procedures,  or  training 
with  conventional  means. 

The  Virtual  Workbench  [65]  and  Image  Overlay  systems 
[66]  both  create  an  immersive  environment,  being  a  mirror 
to  map  a  visual  onto  a  workspace.  Neither,  however,  pro¬ 
vides  force  feedback.  The  Virtual  Environment  Technology 
for  Training  project  [67]  employs  a  three-dof  Phantom  hand 
controller  to  provide  force  on  a  handheld  probe. 

3)  Work  at  SRI:  We  have  begun  to  develop  a  six-dof 
surgical  training  system  using  the  MEDFAST  TSW.  The 
computer-generated  stereo  image  (instead  of  the  remote 
video  scene)  is  reflected  in  the  mirror  and  appears  to  be  su¬ 
perimposed  on  the  operator’s  hands,  creating  an  immersive 
and  realistic  environment.  Tool  handles  held  in  the  user’s 
hands  are  connected  to  left-  and  right-hand  manipulators 
that  each  continuously  measure  tool  position/orientation  and 
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Fig.  11.  Components  of  the  SRI/MusculoGraphics  wound  de¬ 
bridement  demonstration.  (Courtesy  of  MusculoGraphics,  Inc.) 

apply  reaction  forces  and  torques  to  the  handles,  based  on 
modeled  interactions  with  the  tool  tips.  Furthermore,  the 
visual  image  and  tool  locations  are  carefully  registered,  so 
that  the  user  perceives  that  he  is  looking  at  and  moving  the 
simulated  tools  in  the  visual  image. 

SRI  has  been  developing  the  hardware  and  software 
to  interface  the  MEDFAST  TSW  to  an  SGI  graphics 
computer  for  the  purpose  of  developing  a  surgical  simu¬ 
lation  and  training  system  in  surgery.  The  training  system 
will  allow  military  medics  and  surgeons  to  learn  “virtual” 
trauma  management  techniques  for  a  gunshot  wound  to 
the  leg,  as  shown  in  Fig.  11.  The  specific  medical  skills 
that  will  be  taught  include  the  characterization  of  the 
wound,  hemorrhage  control,  assessment  of  muscle  damage, 
intravenous  insertion,  and  debridement.  The  graphics  for 
the  leg  simulation  will  use  a  software  simulation  model 
developed  by  MusculoGraphics,  Inc.,  Evanston,  IL.  Force 
feedback  is  provided  by  the  force  reflection  hardware  and 
software  used  in  the  SRI  telemanipulation  systems  and  is 
based  on  servo  control  digital  signal-processing  hardware. 
The  scope  of  the  simulation  includes  selection  and  motion 
of  two  tools,  contact  detection  between  tool  and  object,  and 
one  deformable  object  (the  leg). 


IV.  Conclusions 

As  a  result  of  the  care  we  have  taken  to  make  our 
telepresence  systems  closely  duplicate  the  operator’s  direct 
visual,  tactile,  and  aural  experience,  we  have  developed 
an  effective  remote  manipulation  environment  with  a  com¬ 
pelling  sense  of  realism.  The  images  of  the  slave  instru¬ 
ments  as  seen  by  the  operator  are  congruent  with  the  master 
controls,  and  the  slave  manipulators  are  seen  to  follow  the 
operator’s  hand  motions  precisely. 

We  have  so  far  been  able  to  develop  telepresence  technol¬ 
ogy  to  the  point  that  our  prototype  four-dof  TMIS  system 
can  speed  the  performance  of  complex  surgical  tasks  by 
about  30%  relative  to  manual  techniques  while  reducing 
operator  stress  and  improving  performance. 
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*  In  the  areas  of  microsurgery  and  remote  open  surgery, 
we  have  demonstrated  excellent  technical  proficiency  and 
tremor  elimination  with  our  telepresence  systems,  but  pro¬ 
cedures  are  still  about  two  to  three  times  slower  than 
direct  hands-on  performance.  Considering,  however,  that 
our  systems  currently  have  fewer  degrees  of  freedom, 
less  bandwidth,  and  more  friction  and  inertia  than  the 
human  hand,  there  is  considerable  opportunity  to  reduce 
the  performance  gap. 

Thus,  when  the  surgical  site  to  be  treated  is  deep  within 
a  patient’s  body  where  our  hands  cannot  reach,  is  too  small 
and  delicate  for  direct  hands-on  work,  or  is  remotely  located 
on  a  space  station  or  at  sea,  telepresence  technology  has  the 
potential  to  provide  a  unique  solution. 
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Telepresence  Surgery 


Medicine  is  entering  a  period  of  pro¬ 
found  technological  transition,  driven 
by  the  need  to  provide  improved  care  to 
more  people  at  lower  cost.  Two  of  the  key 
developments  in  this  transformation  are 
minimal-access  surgery  and  telemedicine. 
SRI  International  has  developed  technol¬ 
ogy  that  significantly  enhances  both  of 
these  developments. 

Minimal-access  surgery,  sometimes 
called  minimally  invasive  or  “closed”  sur¬ 
gery,  encompasses  a  wide  range  of  proce¬ 
dures  that  are  performed  without  major 
incisions.  One  of  these,  laparoscopic  sur¬ 
gery,  has  proved  especially  effective  at 
improving  medical  outcomes  while  reduc¬ 
ing  the  lengths  of  hospital  stay  and  recov¬ 
ery,  consequently  reducing  cost. 
However,  the  range  of  laparoscopic  pro¬ 
cedures  that  can  be  performed  by  most 
general  surgeons  is  limited  by  the  awk¬ 
wardness  of  maneuvering  the  long  instru¬ 
ments,  which  are  fulcrumed  in  the 
abdominal  wall,  and  the  poor  eye-hand 
coordination  afforded  by  the  video  dis¬ 
play,  which  is  typically  positioned  across 
the  operating  table.  By  making  laparo¬ 
scopic  and  other  closed  surgeries  look  and 
feel  like  familiar  “open”  surgery,  SRFs 
Telepresence  technology  significantly  ex¬ 
pands  the  number  and  variety  of  proce¬ 
dures  that  can  be  performed  as  closed 
surgery. 

In  telemedicine,  thus  far,  most  ad¬ 
vances  have  been  diagnostic — for  exam¬ 
ple,  reading  x-rays  at  a  remote  medical 
center  or  mentoring  local  physicians  as 
they  examine  patients.  In  the  future, 
Telepresence  will  enable  expert  surgeons 
at  regional  medical  centers  to  actually  par¬ 
ticipate  in  surgeries  at  local  clinics. 

Teleoperation,  or  remote  manipula¬ 
tion,  has  been  in  use  for  many  years — for 
example,  to  handle  radioactive  materials. 
The  application  of  teleoperation  to  surgery 
has  been  anticipated  and  experimented 
with  for  some  time  [1-4].  However,  in  its 
conventional  form,  teleoperation  does  not 
provide  an  interface  natural  enough  to  en¬ 
able  surgeons  to  operate  in  the  deft  manner 
to  which  they  are  accustomed.  Telepre¬ 
sence  is  enhanced  teleoperation;  the  op¬ 
erator  perceives  that  the  remote  worksite 
is  directly  in  front  of  him  or  her.  The 
Telepresence  surgeon  effectively  uses 
surgical  instruments  that  may  actually  be 
located  in  a  remote  operating  room. 
Grasping  these  instruments,  the  surgeon 


looks  and  reaches  into  the  remote  surgical 
site  and  carries  out  the  operation,  feeling 
the  resistance  of  the  tissues  as  they  are 
touched  by  the  instruments  and  feeling  the 
suture  as  a  knot  is  tied,  just  as  he  would  in 
a  conventional  operation  [5].  In  SRFs 
Telepresence  system,  the  unique  integra¬ 
tion  of  specially  developed  surgical  tele¬ 
manipulators,  feedback  control  systems 
that  provide  touch  sensitivity,  and 
stereographic  video  provides  an  experi¬ 
ence  so  compellingly  real  that,  without 
special  instruction  or  training,  surgeons 
can  carry  out  remote  surgical  procedures 
with  speed,  precision,  and  agility  [6, 7,  8]. 
Using  the  system’s  special  laparoscopic 
manipulators,  difficult  minimal-access 
surgery  can  be  performed  with  relative 
ease,  with  techniques  similar  to  those  of 
conventional  open  surgery.  The  SRI  sys¬ 
tem  differs  from  surgical  robots,  which  are 
preprogrammed  and  move  autonomously 
[9-11],  in  that  the  surgeon  precisely  con¬ 
trols  every  movement  of  the  remote  ma¬ 
nipulators.  Unlike  other  surgical 
telemanipulators  recently  demonstrated 
[12,  13],  which  focus  mainly  on  the  ma¬ 
nipulator,  we  have  developed  a  complete 
surgical  system  integrating  vision,  hear¬ 
ing,  and  manipulation. 

The  System 

SRFs  Telepresence  Surgery  System 
consists  of  two  main  modules:  a  sur¬ 
geon’s  console,  and  a  remote  surgical 
unit  (RSU)  located  at  the  surgical  table. 
As  shown  in  Fig.  1,  the  surgeon  sits  at 
the  console  and  looks  down  into  the 
surgical  field,  a  “virtual  workspace”  rec¬ 
reated  by  a  special  arrangement  of  a  120- 
field/s  stereographic  video  monitor  with  a 
liquid  -crystal  shutter  (the  operator  wears 
passive  polarized  glasses)  and  a  mirror. 
The  stereographic  video  is  generated  by  a 
pair  of  video  cameras  in  the  RSU;  they  are 
positioned  over  the  patient  in  special  rela¬ 
tionship  to  the  surgical  manipulators.  In 
the  laparoscopic  version,  a  stereo  laparo¬ 
scope  (available  from  several  manufactur¬ 
ers)  is  substituted  for  the  camera  pair. 

As  shown  in  Fig.  2,  the  surgeon  reaches 
under  the  mirror  and  grasps  two  surgical- 
instrument  handles  (hemostats  with  the 
latches  and  jaws  removed)  that  operate 
two  master  manipulators.  Each  master  di¬ 
rects  the  movement  of  a  slave  manipulator 
at  the  RSU,  shown  in  Fig.  3,  by  means  of 
a  highly  responsive,  force-reflecting 
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1.  Surgeon  looks  down  into  the  apparent  workspace,  a  stereo-video  recreation  of  the 
actual  workspace  at  the  left.  The  remote  manipulators  appear  to  emerge  from  the 
hand  controls,  which  are  not  seen  by  the  surgeon.  The  master  and  slave  move  in  per¬ 
fect  synchrony  and  reflect  forces  realistically. 


2,  The  latest  surgeon’s  console  provides 
two  hand-control  masters  with  surgical- 
instrument  handles,  a  stereographic 
view  into  the  surgical  site,  and  a  micro¬ 
phone  and  speakers  for  communicating 
with  the  surgical  assistant  or  nurse  in 
the  operating  room.  Looking  straight 
ahead,  the  surgeon  sees  a  panoramic 
view  across  the  table,  on  three  side-by- 
side  LCDs. 


3.  The  remote  surgical  unit  (RSU)  com¬ 
prises  two  slave  manipulators  with  inter¬ 
changeable  surgical  instruments,  two 
video  cameras  positioned  correctly  with 
respect  to  the  manipulators  and  canted 
in  for  stereo  convergence  in  the  middle 
of  the  operating  field,  microphones,  and 
a  trio  of  video  cameras  (not  shown)  that 
produce  a  panoramic  view  of  the  operat¬ 
ing  room  from  the  surgeon’s  perspective. 


servo-controller.  Affixed  to  the  ends  of  the 
manipulator  arm  are  interchangeable  sur¬ 
gical  instruments — forceps,  needle  driv¬ 
ers,  bowel  graspers,  scalpels,  and  cautery 
tips,  as  shown  in  Fig.  4.  The  surgeon  sees 


the  instruments  in  the  stereographic  im¬ 
age — they  appear  to  be  emerging  from  the 
handles  in  his  hands.  When  he  moves  his 
hand  controls,  the  instruments  move  as  if 
they  were  rigidly  attached  to  the  controls. 


Sounds  from  the  instruments  or  the  sur¬ 
geon’s  assistants  are  picked  up  by  stereo 
microphones  and  relayed  to  speakers  in 
the  surgeon’s  console.  To  the  surgeon,  it 
looks  and  feels  (and  sounds)  as  if  he  is 
performing  the  task  right  before  his  eyes 
with  conventional  surgical  instruments  in 
his  hands.  The  strength  of  this  perception 
enables  the  surgeon  to  carry  out  complex 
tasks  with  quick,  precise  motions. 

The  masters  and  slaves  are  lightweight, 
responsive,  well-balanced,  and  gravity- 
compensated.  Friction  has  been  mini¬ 
mized  by  reducing  the  stages  of  gearing, 
using  low-friction/low-inertia  actuators, 
and  using  low-friction  bearings  through¬ 
out.  The  usual  drawbacks  of  inertia  and 
friction — degraded  operator  performance 
and  fatigue  after  prolonged  use — have 
been  largely  eliminated.  The  bilateral  con¬ 
trol  system  provides  for  scaled  motion  and 
force  reflection,  which,  along  with  scaled 
video,  enable  the  surgeon  to  perform  mi¬ 
crosurgery  with  normal  hand  motions. 
The  current  open-surgery  manipulators 
have  five  degrees  of  freedom,  with  force 
feedback  in  each  axis  (including  opening 
and  closing  the  jaws  of  graspers  or  scis¬ 
sors).  More  dexterous  manipulators  and 
many  other  improvements  are  being  in¬ 
corporated  into  the  system  for  specific 
applications,  as  described  below. 

Remote  Surgery  on  the  Battlefield 

Potentially,  Telepresence  surgery  can 
make  a  significant  contribution  to  the  man¬ 
agement  of  combat  trauma.  Nine  out  of  ten 
combat-related  deaths  from  nonfatal  inju¬ 
ries  occur  in  the  combat  zone,  prior  to 
evacuation,  the  majority  from  loss  of 
blood.  Many  of  these  lives  could  be  saved 
if  surgical  treatment  were  available  within 
one  hour  of  injury — dubbed  the  “Golden 
Hour”  by  Army  Surgeon  General  Alcide 
Lanoue.  However,  in  modern  warfare,  the 
front  line  may  move  forward  so  quickly 
that  mobile  army  surgical  hospitals 
(MASH  units)  cannot  get  established  close 
enough  to  the  combat  zone  to  be  effective. 
Cunently,  the  only  care  available  to  the 
seriously  wounded  soldier  within  the  com¬ 
bat  zone  is  that  provided  by  the  field  medic, 
whose  goal  is  to  stabilize  him  and  prepare 
the  victim  for  evacuation  [14]. 

To  provide  surgical  services  in  the  zone 
of  combat,  SRI  is  developing  a  battlefield 
version  of  its  Telepresence  Surgei^  System. 
This  version  has  been  installed  in  a  mobile 
surgical  vehicle,  dubbed  the  medical  emer¬ 
gency  forward  area  surgical  telepresence 
(MEDFAST),  envisioned  to  be  deployed  as 
illustrated  in  Fig.  5.  After  first  aid  has  been 
administered  by  the  medic,  the  wounded 
soldier  in  need  of  immediate  surgery  is 
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4.  An  array  of  interchangeable  surgical  instruments. 


JSTARS  relay 


5.  From  a  position  safely  behind  the  lines,  the  Telepresence  combat  surgeon  of  the 
future  will  carry  out  surgeries  in  mobile  surgical  vehicles  operating  throughout  the 
zone  of  combat,  with  the  assistance  of  medics  assigned  to  each  vehicle.  First  field 
tests  will  employ  a  point-to-point  microwave  link  to  carry  video,  servo,  audio,  and 
patient  monitoring  information,  which  eventually  may  be  relayed  by  low-earth-or- 
bit  satellites. 


brought  to  the  MEDFAST  and  put  on  the 
operating  table.  The  Telepresence  combat 
surgeon  carries  out  only  a  limited  spec¬ 
trum  of  procedures.  These  are  primarily 
for  stabilizing  wounded  soldiers,  so  that 
they  can  be  evacuated  when  conditions 
permit,  and  for  treating  conditions  that,  if 
deferred,  would  seriously  affect  recovery. 
The  most  threatening  conditions  are  air¬ 
way  obstructions,  sucking  chest  wounds, 
and  profuse  bleeding.  The  surgeon  can 


remotely  explore  for  and  repair  injuries  to 
major  vessels  and  abdominal  and  pelvic 
organs,  remove  bone  fragments  and  for¬ 
eign  material,  and  debiide  damaged  tis¬ 
sues.  The  wound  can  then  be  packed  and 
dressed;  wound  closure  is  left  to  the  next 
echelon  of  care.  Telepresence  surgery 
greatly  augments  the  trauma  care  that  the 
medic  alone  can  provide. 

This  concept  was  demonstrated  in  June 
1993  during  a  medical  field  exercise  held 


at  Ft.  Gordon  in  Augusta,  GA.  In  a  tent. 
Army  surgeons,  seated  at  an  early  version 
of  our  console  operated  on  an  “injured 
soldier” — actually,  a  life-like  mannequin 
with  pig  intestines  in  its  abdominal  cavity. 
A  medic  at  the  patient’s  side  assisted  the 
surgeon.  The  patient,  the  RSU,  and  the 
medic  were  located  in  a  transportable  op¬ 
erating  room  (OR)  30  meters  from  the  tent. 
Cables  carried  the  control,  video,  and 
audio  signals  between  the  units  (a  high- 
bandwidth,  two-way  radio  link  would  be 
required  in  an  actual  combat  situation).  At 
Ft.  Gordon,  only  a  single  master-slave 
combination  was  used;  nevertheless,  the 
potential  was  evident  and  the  surgeons 
were  enthusiastic. 

Encouraged  by  the  Ft.  Gordon  results, 
we  began  preparation  for  a  much  more 
realistic  demonstration,  in  which  two- 
handed  surgery  would  be  carried  out  in  a 
combat-ready  vehicle  outfitted  as  a  mobile 
OR.  The  new  surgeon’s  console  (Fig.  2) 
was  constructed  and  outfitted  [15].  This 
version  incorporates  two  hand-control 
masters  with  surgical-instrument  handles, 
a  stereographic  monitor  viewed  with  a 
mirror,  a  microphone  and  speakers  for 
communicating  with  the  surgical  assistant 
or  nurse  in  the  OR,  and  three  side-by-side 
liquid  crystal  displays  (LCDs)  that  pro¬ 
vide  a  panoramic  view  across  the  operat¬ 
ing  table,  allowing  the  surgeon  to  see  the 
medic  and  any  activity  in  the  OR.  The  new 
OR,  a  577  tracked,  armored  vehicle,  was 
equipped  with  a  special  operating  table,  an 
electrically  controlled  gantry  for  position¬ 
ing  the  RSU  over  the  patient,  and  other 
equipment,  as  shown  in  Fig.  6. 

This  equipment  was  transported  to 
Washington,  D.C.,  in  October  1994,  and 
demonstrated  at  the  annual  convention  of  the 
Association  of  the  U.S.  Army.  Still  linked 
by  a  cable,  the  two  units  were  situated  160 
meters  apart — the  577  in  the  hotel  parking 
lot  and  the  surgeon’s  console  in  a  third-floor 
exhibition  hall.  On  the  operating  table  lay  a 
soldier  mannequin  with  lacerated  pig  intes¬ 
tines  and  liver  in  its  abdominal  cavity,  along 
with  bits  of  shrapnel,  and  a  simulated  bleed¬ 
ing  artery  squirting  blood  (activated  surrep¬ 
titiously  by  the  medic).  This  is  representative 
of  the  type  of  abdominal  wound  suffered  in 
combat  [16], 

Surgeons,  and  many  others,  had  the 
opportunity  to  sit  at  the  console,  suture  the 
lacerations,  remove  shrapnel,  and  expose 
the  “bleeder,”  which  the  medic  then 
clamped.  Surgeon  and  medic  worked  to¬ 
gether  as  if  they  were  standing  across  the 
table  from  each  other.  For  example,  the 
medic  passed  the  suture  to  the  surgeon  and 
then  provided  countertension  on  the  tis¬ 
sues  while  the  surgeon  sewed.  Surgeons 
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6.  Operating  room  in  a  tracked,  armored  vehicle,  (a)  The  XM577A3  armored  tacti¬ 
cal  command  and  control  system  vehicle,  which  was  modified  for  this  project  by  Fos¬ 
ter-Miller,  Inc.,  in  cooperation  with  United  Defense  Limited  Partnership,  (b) 
interior  of  the  577,  showing  the  operating  table,  surgical  manipulators,  cameras  and 
lights,  the  medic,  and  auxiliary  surgical  and  support  equipment.  The  “patient”  was 
a  dummy  with  pig  intestines,  shrapnel,  and  a  simulated  bleeding  artery  in  its  ab¬ 
dominal  cavity. 


were  able  to  run  suture  lines  and  do  instru¬ 
ment  ties  with  high  dexterity  and  speed. 

The  general  consensus  was  that  the 
feasibility  of  Telepresence  surgery  on  the 
battlefield  had  been  demonstrated,  al¬ 
though  a  number  of  improvements  were 
needed.  Preparation  of  the  system  has  be¬ 
gun  for  actual  field  trials.  Three  notable 
features  are  being  added:  (1)  a  pair  of 
seven-degree-of-freedom  hand  controls 
and  manipulators,  (2)  a  wireless  commu¬ 
nication  system,  and  (3)  a  broader  array  of 
surgeon-interchangeable  instruments. 

Telepresence  Laparoscopic  Surgery 

Laparoscopic  surgery  has  almost  to¬ 
tally  replaced  open  surgery  for  cholecys¬ 
tectomy  (gallbladder  removal).  However, 
few  surgeons  are  able  to  progress  to  more 
complex  laparoscopic  procedures,  such  as 
bowel  and  esophageal  surgery,  because  of 
the  inadequacy  of  current  instrumentation. 
In  the  U.S.  alone,  there  are  perhaps  four 
million  open  surgical  procedures  per¬ 
formed  each  year  that  could  be  converted 
to  closed  procedures  if  better  surgical 
technology  were  available.  The  benefits 
would  be  substantial,  as  has  been  demon¬ 
strated  for  cholecystectomy:  lower  costs, 
shorter  hospital  stays,  much  quicker  re¬ 
covery,  less  infection,  and  less  cosmetic 


damage.  The  problem:  laparoscopic  sur¬ 
gery  entails  awkwardly  maneuvering  long 
instruments  that  are  inserted  through  can- 
nulae  in  the  abdominal  wall.  The  wall  acts 
as  a  fulcrum,  so  that  the  direction  of  mo¬ 
tion  of  the  instrument  tip  is  opposite  that 
of  the  surgeon’s  hand,  and  the  magnitude 
of  the  tip  motion  depends  on  the  extent  to 
which  the  instrument  has  been  inserted. 
The  surgeon  watches  the  instrument  tip 
and  the  internal  organs  on  a  video  screen 
across  the  OR  table,  an  arrangement  that 
makes  eye-hand  coordination  extremely 
difficult.  This  combination  of  activities  is 
totally  unnatural;  every  motion  must  be 
deliberate  (rather  than  spontaneous,  as  it 
is  in  open  surgery).  Moreover,  the  instru¬ 
ments  provide  little  force  feedback,  which 
is  very  important  to  the  surgeon.  The  com¬ 
bined  effect  is  to  make  suturing,  knot  ty¬ 
ing,  and  many  other  usually  simple 
maneuvers  exceedingly  difficult  to  do  la- 
paroscopically. 

Telepresence  overcomes  these  barriers 
to  effective  laparoscopic  surgery.  Al¬ 
though  SRTs  Telepresence-operated  la¬ 
paroscopic  instruments  (as  also 
conventional  laparoscopic  instruments) 
are  inserted  through  cannulae  in  the  ab¬ 
dominal  wall,  the  surgeon’s  perception  is 
that  of  using  the  traditional  instruments  in 
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an  open  surgical  field.  The  Telepresence 
laparoscopic  surgeon  grasps  the  open-sur¬ 
gery  instrument  handles,  looks  down  into 
the  actual  surgical  field  (via  a  3-D  video 
laparoscope),  and  operates.  Now,  his 
hands  are  effectively  inside  the  abdomen, 
on  the  correct  side  of  the  fulcrum.  The 
instrument  tips,  as  viewed  by  the  surgeon, 
move  as  if  they  were  rigidly  connected  to 
the  handles  in  his  hands.  As  the  instru¬ 
ments  touch  tissue,  the  surgeon  feels  the 
resistance  through  the  handle.  Unlike  con¬ 
ventional  laparoscopic  surgery,  which  re¬ 
quires  extensive  additional  training  and 
continued  practice,  Telepresence  surgery 
thus  requires  little  additional  training. 

Preliminary  dexterity  experiments  in¬ 
volving  simple  maneuvers,  such  as  trans¬ 
ferring  beads  from  wire  to  wire  and 
cannulizing  one  plastic  tube  with  another, 
have  also  been  conducted.  Comparisons 
were  made  between  ( 1 )  free-hand  opera¬ 
tion  using  a  hemostat,  (2)  the  use  of  a 
conventional  fulcrumed  laparoscopic 
grasper  viewed  with  a  monoscopic  video 
laparoscope,  and  (3)  Telepresence  opera¬ 
tion,  using  an  open-surgery  Telepresence 
manipulator  with  both  monoscopic  and 
stereoscopic  viewing  [I?].  The  results 
confirmed  expectations.  Three  subjects 
took  7  to  1 5  times  longer  with  the  conven¬ 
tional  laparoscopic  instruments  than  with 
Telepresence;  in  these  tests,  the  latter  was 
almost  as  effective  as  free-hand  manipu¬ 
lation.  SRTs  laparoscopic  Telepresence 
equipment  is  now  undergoing  evaluation 
in  animal  surgery. 

Telepresence  laparoscopic  surgery 
will  result  in  significantly  lower  health 
care  costs.  OR  time  for  complex  proce¬ 
dures  such  as  colon  resections  and  other 
bowel  surgeries,  Nissen  fundoplication, 
and  laparoscopically  assisted  vaginal  hys¬ 
terectomy  will  be  only  marginally  higher 
than  those  of  an  open  procedure — much 
less  than  for  laparoscopic  surgery  as  cur¬ 
rently  practiced.  Costs  associated  with  the 
hospital  stay  and  return  to  normal  function 
will  be  the  same  as  for  current  laparo¬ 
scopic  procedures — much  lower  than  for 
open  surgery.  Telepresence  laparoscopic 
surgery  should  combine  the  best  of  both 
worlds.  It  is  expected  that  patient  demand 
will  drive  the  adoption  of  other  laparo¬ 
scopic  procedures  and  that  surgeons  will 
eagerly  respond  to  this  demand  once  they 
have  a  technology  that  allows  them  to 
confidently  undertake  the  procedures. 

The  Future  of  Telesurgery 

Telepresence  clearly  has  the  potential 
for  bringing  to  outlying  clinics  the  skills 
of  expert  specialists  at  regional  medical 
centers.  In  a  cost-conscious  era,  better  use 
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paroscopic  surgery.  The  surgeon  will  also  control  the  laparoscope  position. 


of  specialist  skills  and  lower  patient  trans¬ 
port  costs  are  both  appealing.  The  rapid 
development  of  diagnostic  telemedicine 
demonstrates  that  a  real  need  is  being  met. 
Recent  animal  studies  have  demonstrated 
that  microsurgical  procedures,  such  as 
vascular  repair,  can  be  effectively  per¬ 
formed  by  Telepresence  [18].  Techno¬ 
logically,  there  seems  to  be  no 
fundamental  barrier  to  implementing  tele- 
surgery,  although  achieving  fail-safe  per¬ 
formance  of  both  the  sophisticated 
equipment  and  the  communication  link  is 
a  significant  task,  and  propagation  delays 
may  present  a  challenge  at  distances 
greater  than  a  few  hundred  kilometers. 
However,  many  important  clinical  consid¬ 
erations  will  bear  heavily  on  the  practical¬ 
ity  of  this  telesurgery  in  the  civilian  sector. 

As  illustrated  in  Fig.  8,  the  appropriate¬ 
ness  of  Telepresence  surgery  for  a  particu¬ 
lar  patient  at  a  particular  clinic  will  depend 
upon  the  nature  of  the  procedure  to  be 
performed  and  the  clinic’s  in-house  capa¬ 
bilities.  If  the  procedure  is  minor  and  can 
be  accomplished  by  someone  on  site  (pos¬ 
sibly  with  mentoring  from  a  medical  cen¬ 
ter),  then  telesurgery  would  not  be 
indicated.  If  no  one  on  site  is  competent  to 
assist  the  surgeon,  or  if  the  clinic  lacks  the 
capability  to  deal  with  complications  or 
postoperative  care,  then  telesurgery  would 
not  be  indicated.  What  lies  in  between  is 
telesurgery’s  potential  field  of  play.  A 
high  enough  case  load  would  justify  the 
cost  of  the  necessary  capital  equipment, 
training,  and  maintenance. 


Comments  and  Conclusions 

SRI  has  established  that  precise  surgi¬ 
cal  procedures  can  be  carried  out  with 
Telepresence.  Within  this  decade, 
Telepresence  surgery  may  become  an  es¬ 
tablished  component  of  a  new,  technologi¬ 
cally  enhanced  mode  of  cost-effective 
health  care  delivery. 

Because  of  the  substantial  medical  and 
financial  benefit  that  Telepresence  will 
bring  to  laparoscopic  and  other  minimally 
invasive  procedures,  its  first  routine  use  is 
expected  to  be  within  the  individual  hos¬ 
pital  and  surgical  center.  Telepresence 
surgical  procedures  have  been  demon¬ 
strated  over  distances  of  160  meters,  and 
we  are  working  toward  demonstrating  ani¬ 
mal  surgeries  over  a  much  greater  distance 
by  using  microwave  and  fiber-optic  links. 
Mobile  Telepresence  surgical  equipment 
may  prove  especially  effective  for  trauma 
care  in  combat  and  in  civilian  disasters. 

Because  the  stereographic  images  and 
the  motions  and  forces  can  be  readily  scaled, 
microsurgeries  of  all  kinds  can  be  made 
easier  with  this  technology.  Arthroscopic 
procedures,  especially  difficult  surgery  on 
joints  such  as  the  wrist  and  shoulder,  are 
good  Telepresence  candidates.  In  neurosur¬ 
gery,  excision  of  pituitary  adenomas  could 
benefit  from  the  increased  dexterity  and  tac¬ 
tile  sensitivity  of  Telepresence.  Eventually, 
neurosurgery  under  real-time  magnetic 
resonance  imaging  could  be  performed  by  a 
Telepresence  surgeon  operating  on  a  three- 
dimensional  reconstruction  that  is  immedi¬ 
ately  updated  to  show  the  actual  tissue 
changes  caused  by  the  telemanipulated  in¬ 
struments  (which,  of  course,  would  need  to 
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be  nonmagnetic).  One  application  of  this 
combination  of  technologies  might  be  the 
removal  of  colloid  cysts  from  the  brain. 
Gaining  access  to  these  cysts  without  dis¬ 
turbing  surrounding  tissues  is  now  diffi¬ 
cult.  Computer-controlled  exclusion 
zones  could  be  established  on  the  image 
prior  to  surgery  to  constrain  the  instru¬ 
ment’s  path. 
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